Data Engineer ID43228AgileEngine • Houston, TX, us
Data Engineer ID43228
AgileEngine • Houston, TX, us
30+ days ago
Job type
- Full-time
- Quick Apply
Job descriptionYou must be authorized to work for ANY employer in the US (e.g., Green card holders, TN visa holders, GC EAD, H4 EAD, U4U with EAD), as we are unable to sponsor or take over employment visa sponsorship at this time; 3+ years of professional experience with Python ; 3+ years of professional experience in a Data Engineering role; Proficiency in programming languages commonly used in data engineering such as Python , SQL , and optionally Scala for working with data processing frameworks like Spark and libs like Pandas; Proficiency in designing, deploying, and managing data pipelines using Apache Airflow for workflow orchestration and scheduling; Ability to design, develop, and optimize ETL processes to move and transform data from various sources into the data warehouse, ensuring data quality, reliability, and efficiency; Knowledge of big data technologies and frameworks such as Apache Spark for processing large volumes of data efficiently; Extensive hands-on experience with various AWS services relevant to data engineering, including but not limited to Amazon MWAA, Amazon S3, Amazon RDS, Amazon EMR, AWS Lambda, AWS Glue, Amazon Redshift, AWS Data Pipeline, Amazon DynamoDB; Deep understanding and practical experience in building and optimizing cloud data warehousing solutions; Ability to monitor data pipelines, identify bottlenecks, and optimize data processing and storage for performance and cost-effectiveness; Excellent communication skills to collaborate effectively with cross-functional teams including data scientists, analysts, software engineers, and business stakeholders; Bachelor’s degree in computer science / engineering or other technical field, or equivalent experience; Upper-intermediate English level. Familiarity with the fintech industry, understanding of financial data, regulatory requirements, and business processes specific to the domain; Documentation skills to document data pipelines, architecture designs, and best practices for knowledge sharing and future reference; GCP services relevant to data engineering; Snowflake; OpenSearch, Elasticsearch; Jupyter for analyze data; Bitbucket, Bamboo; Terraform. Professional growth : Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps. Competitive compensation : We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities. A selection of exciting projects : Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands. Flextime : Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
Job Description
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
As a Middle Data Engineer , you’ll build and optimize ETL pipelines and cloud data solutions that provide reliable insights for data scientists and analysts. You’ll tackle complex data challenges, collaborate with cross-functional teams, and grow your expertise in Python, Airflow, Spark, and AWS in a dynamic, innovative environment.
WHAT YOU WILL DO
- Build and support ETL pipelines;
- Monitor data pipelines, identify bottlenecks, optimize data processing and storage for performance and cost-effectiveness;
- Collaborate effectively with cross-functional teams including data scientists, analysts, software engineers, and business stakeholders;
- Working with Terraform to build AWS infrastructure;
- Analyze sources and build Cloud Data Warehouse and Data Lake solution.
MUST HAVES
NICE TO HAVES
PERKS AND BENEFITS
Create a job alert for this search
Data Engineer • Houston, TX, us
Related jobs
About the job Marketing Data Engineer.DIVIETO Ristorante Group is one of the fastest-growing upscale restaurant brands in the United States, known for delivering exceptional Italian cuisine and unf...Show more
All associates are guided by Our Values.Our Values are the unifying foundation of our companies.We strive to ensure that every decision we make and every action we take demonstrates Our Values.We b...Show more
Southern New Hampshire University is a team of innovators.Individuals who believe in progress with purpose.Since 1932, our people-centered strategy has defined us - and helped us grow a team that n...Show more
About Neudesic, an IBM Company : .Passion for technology drives us, but it’s innovation that defines us.From design to development and support to management, Neudesic offers decades of experience, pr...Show more
Data Engineer – Full-Time Location : .Sugar Land, TX Salary Range : $130,000 – $170,000 (DOE) About CG Infinity CG Infinity is a technology consulting firm founded in 1998.We deliver tailo...Show more
Senior Azure Databricks Engineer.Our mission is to improve outcomes for Chronic Care patients who are dependent on multiple daily medications.
Our proprietary and vertically integrated EMR technolog...Show more
Marketing Data Engineer
Anarust, Inc • Houston, TX, United States
Full-time
Last updated: 3 days ago • Promoted
Lead Data Engineer
The Friedkin Group • Houston, TX, United States
Full-time
Last updated: 30+ days ago • Promoted
Principal Data Platform Engineer
Southern New Hampshire University • Houston, TX, United States
Full-time
Last updated: 6 days ago • Promoted
Senior Data Engineer
Neudesic, an IBM Company • Houston, TX, United States
Full-time
Last updated: 18 days ago • Promoted
Data Engineer- Houston Based
CG Infinity • Houston, TX, US
Full-time
Quick Apply
Last updated: 30+ days ago
Senior Databricks Engineer (Houston)
Altea Healthcare • Houston, TX, US
Part-time
Last updated: 10 days ago • Promoted