Sr. Data EngineerTalent Portus • OH, United States

Sr. Data Engineer

Talent Portus • OH, United States

5 days ago

Job type

Full-time

Quick Apply

Job description

Need candidate to go F2F interview

Data Engineer

MUST BE LOCAL TO COLUMBUS AND WORK HYBRID

Job Description :

The Data Engineer will implement and maintain key data pipelines, focusing on data ingestion, replication, and continuous integration for batch and streaming data.

This position will leverage their proficiency in programming, ETL processes, and cloud data technologies to develop, maintain, test, and deploy scalable data pipelines while ensuring data quality and reliability.

This role will focus on building robust, maintainable, accessible, cost effective, and observable data pipelines between source systems and our Snowflake data lake.

Essential Functions (% time) :

Build scalable ETL pipelines to automate data ingestion, for both batch and streaming data processes, into a centralized data lake in Snowflake. Leverage some off-the-shelf connectors and some custom python solutions to deliver these pipelines (20%.)
Develop and deploy orchestration solutions to facilitate near real time data availability for strategically important data sets (20%.)
Enhance monitoring and alerting systems to proactively identify and resolve issues, minimizing downtime and ensuring data consistency, resiliency, and availability (20%.)
Build security and access controls into data pipelines to enable granular Role Based access to enterprise data, including security at row and column levels (15%.)
Partner with Software Engineering teams to understand their development roadmap and ensure that data pipelines are maintained and prepared for any schema or business logic changes (15%).
Provide support to Analytics Engineers tasked with transforming and interpreting raw source system data into dimensional models (10%.)

Qualifications : Education :

Bachelor's Degree in computer science, engineering, analytics, or related field.

Work Experience :

1-3 years hands on experience in a data engineering or related role.

Knowledge, Skills, Abilities (Proficiency)

High Proficiency :

Proficiency with at least one cloud based ELT technology (Fivetran, AWS Kinesis, Snowpipe, Airbyte)

Experience building and operating highly available, self healing systems for data extraction, ingestion, and continuous integration and deployment.

Experience with cloud-based data warehousing and data lake solutions like Snowflake, Redshift, and BigQuery

Familiarity with one or more DevOps tools (Git, Jenkins, CI / CD, Jira)

Medium Proficiency :

Proficiency in at least one general purpose interpreted programming language (Python preferred)

Ability to think strategically and provide recommendations utilizing modern cloud architectural components based on business needs with a focus on simplicity and maintainability.

Experience building data pipelines to ingest data from systems built on legacy technologies (IBM VSAM) into modern cloud-based data lake tooling (Snowflake).

Ability to orchestrate complex workflows using tools like Airflow or Dagster Medium Expertise with SQL, database design / structure, and data structures (star, snowflake schemas, de-normalized designs).

Experience with script-based data transformation tools (e.g., dbt, Talend) and version control tools.

Desire and ability to collaborate with cross-functional teams of software engineers, product managers, and business stakeholders.

Best Regards,

Mohd Amzad

Technical Recruiter || Talent Portus

Email : mohd.amzad@talentportus.com

LinkedIn : linkedin.com / in / mohd-amzad

Create a job alert for this search

Sr Data Engineer • OH, United States