AWS Data Engineer

Role : AWS Data Engineer Location : Torrance, CA Duration : Longterm Contract Job Description : 5+ years of experience in data engineering, database design, ETL processes, and data warehousing. 3+ years of experience with AWS tools and technologies (S3, EMR, Glue, Athena, RedShift, RDS, Spectrum and Airflow) 2+ years of experience with CI / CD tools. Strong knowledge of data storage and processing technologies, including databases and data lakes based distributed computing frameworks (e.g., Hadoop, Spark). 3+ in programming languages such as Python, Java, or Scala. Nice to have Informatica Cloud experience (IDMC) Detailed Responsibilities : Develop and Maintain Data Integration Solutions : o Design and implement data integration workflows using AWS Glue / EMR,AWS MWAA(Airflow), Lambda, Redshift o Demonstrate proficiency in Pyspark, Apache Spark and Python for data processing large datasets o Ensure data is accurately and efficiently extracted, transformed, and loaded into target systems. Ensure Data Quality and Integrity : o Validate and cleanse data to maintain high data quality. o Ensure data quality and integrity by implementing monitoring, validation, and error handling mechanisms within data pipelines Optimize Data Integration Processes : o Enhance the performance, optimization of data workflows to meet SLAs, scalability of data integration processes and cost-efficiency on AWS cloud infrastructure. o Identify and resolve performance bottlenecks, fine-tuning queries, and optimizing data processing to enhance Redshift's performance o Regularly review and refine integration processes to improve efficiency. Support Business Intelligence and Analytics : o Translate business requirements to technical specifications and coded data pipelines o Ensure timely availability of integrated data for business intelligence and analytics. o Collaborate with data analysts and business stakeholders to meet their data requirements. Maintain Documentation and Compliance : o Document all data integration processes, workflows, and technical & system specifications. o Ensure compliance with data governance policies, industry standards, and regulatory requirements.