Data Engineer

Data Engineer

Brighton, MA (Onsite)

Long Term

Key Responsibilities :

• Design, develop, and implement end-to-end big data solutions across the enterprise.
• Develop and maintain big data applications using Apache Spark , Scala , AWS Glue , AWS Lambda , SNS / SQS , and CloudWatch .
• Build and optimize data pipelines, ensuring high performance, scalability, and reliability.
• Collaborate with cross-functional teams to design and document integration and application technical designs.
• Conduct peer reviews of functional and design documentation to maintain high-quality standards.
• Develop, configure, and perform unit testing and code reviews to ensure coding best practices.
• Troubleshoot and resolve complex issues during testing phases and identify root causes efficiently.
• Perform performance testing and optimize system performance.
• Manage and maintain SQL-based databases , preferably Amazon Redshift .
• Utilize Snowflake for data warehousing (experience in Snowflake is an added advantage).
• Implement ETL / ELT processes and ensure data quality across various systems.
• Work with Git repositories and manage CI / CD deployment pipelines for continuous integration and delivery.
• Provide production support , including troubleshooting and environment tuning.
• Ensure adherence to best practices and technical standards throughout the project lifecycle.

Required Skills & Experience :