Position : Senior Big Data Engineer Location : Costa Mesa, CA Duration : 12+ Months Our team is responsible of dealing with huge amount of data and also have an integration from various systems, create a required pipeline, and schedule, also team is dealing with steaming data for Realtime applications.
What you will do here :
- Design & build all batch / Realtime components / data pipelines lines / integrations for our MLOps platform on AWS.
- Collaborate with geographically distributed cross-functional teams.
- Participate in on-call rotation with the rest of the team to handle production incidents.
About the Role :
Big Data Experience 4+ years experience.Strong in Python / SCALA / Java programmingStrong understanding / experience in SPARK.Good exposure on Kafka / streaming.AWS Experience is a must EMR, ECS, Elastic Search, S3, Cloud Watch, Lambda, Step Functions,Good communication skills to work with various stakeholders on operational support workDEVOPS Experience is a must CI / CD, Jenkins Build, Docker, ECR RepoNice to have skills :
Experience with AWSExperience Terraform is good to have.Experience with various Python packaging options such as Wheel, PEX or Conda.Experience with various automated testing framework is plus.