Benefits Summary
- Flexible and hybrid work arrangements
- Paid time off / Paid company holidays
- Medical plan options / prescription drug plan
- Dental plan / vision plan options
- Flexible spending and health savings accounts
- 401(k) retirement savings plan with a Roth savings option and company matching contributions
- Educational assistance program
Overview
The Sr. Data Engineer requires expertise in DataOps and is responsible for guaranteeing dependable, scalable and effective data processes and solutions by combining components, functions and skills from data engineering, software engineering and IT Operations. This role is critical to AM Best's goal of developing, maintaining and scaling a sophisticated platform for data and analytics.
Responsibilities
Create and support scalable data solutions :
Build high performance data systems including databases, APIs, and data integration pipelinesImplement a metadata-driven architecture and infrastructure as code approach to automate and simplify the design, deployment, and management of data systemsFacilitate the adoption of data and software engineering best practices, including code review, testing, and continuous integration and delivery (CI / CD)Develop and establish a data governance frameworkContinuous monitoring of process performance and implement improvements for efficiency including fine-tuning existing ETL processes, optimizing queries, or refactoring codeAssess and make optimal use of cloud platforms and technologies, especially Azure and Databricks to enhance system architectureImplement data quality checks and build processes to identify and resolve data issuesCreate and maintain documentation for data architecture, standards, and best practicesContribute designs, code, tooling, testing, and operational supportIncrease operational efficiency :
Identify opportunities for process optimization and automation to enhance data operations efficiencyLeadership and innovation :
Provide technical leadership to the data engineering team and actively lead design discussionsEnsure that the new data infrastructure remains modern and efficient by staying knowledgeable of the latest tools, technologies, and methodologies in the data and analytics spaceQualifications
7 plus years of experience working as a Data Engineer, Data Architect, Database Developer or similar rolesBachelor's Degree in Computer Science, Data Science, Engineering or related fieldSkills
7+ years of experience in programming languages such as Python, Scala, Java, Rust or similar7+ years of experience in Oracle OLTP database design and development7+ years of experience in building ETL / ELT data pipelines using Apache Spark, Airflow, dbt or similar7 years of experience in developing APIs that include REST, SOAP, RPC, GraphQL or similar7 years of experience in OLAP / data warehouse design and development using dimensional (Kimball star-schema), data vault or Inmon methodologies5 years of experience in NoSQL database design and development preferably on MongoDB or similar5 years of experience working with message broker platforms like RabbitMQ, Apache Kafka, Red Hat AMQ or similar5 years of experience working with Azure or AWS cloud-native technologies / servicesWorking knowledge of CI / CD automation tools or services like Jenkins, Ansible, Chef, Puppet or similarWorking knowledge of containerized platform or services like Docker, Kubernetes or similarMust be hands-on, have a passion for data and problem solving, and an innovative mind-set to help deliver results across projectsPluses :
Experience working with Databricks data intelligence platformExperience working with business intelligence tools / platforms like PowerBI, Cognos, Tableau or similarExperience working with caching databases : Memcache,Redis or similarExperience working with search databases and platforms : Elasticsearch, Apache Solr or similarExperience working with data modeling tools : ERwin, ERStudio, Navicat, SqlDBM or similarKnowledge of data governance tools : Colibra, Alation or similarKnowledge of master data management tools : Informatica, TIBCO, Riversand, Ataccama or similar