Platform Automation Engineer

Job Title : Platform Automation Engineer

Job Type : Contract to hire

Duration : Months contract to hire

Location : Plano, TX

% Onsite role

About the Role :

We are seeking an innovative and highly skilled Senior Platform Automation Engineer to drive the design, development, and maintenance of advanced platform-level automation and integrations for our Databricks data lake on AWS. This role is crucial for enhancing the efficiency, scalability, reliability, and security of our data platform, empowering our data science and analytics teams through cutting-edge automation and continuous innovation.

Responsibilities :

• Lead the design, development, and implementation of innovative platform automation solutions for the Databricks data lake on AWS.
• Automate the provisioning, configuration, and management of core AWS services (S, IAM, EC, VPC, Lambda, CloudWatch, SNS, SQS, ELB, Route) using Infrastructure as Code (Terraform).
• Develop robust automation scripts and tools using Python for Databricks workspaces, clusters, jobs, and security configurations, leveraging Databricks APIs and CLI.
• Design, implement, and maintain CI / CD pipelines (, GitHub Actions) for continuous delivery of infrastructure, platform, and automation code.
• Implement and enforce security best practices, compliance, and cost optimization strategies across the AWS and Databricks data platform.
• Integrate and automate the management of AWS data services (Redshift, DynamoDB, Glue, Athena) within the Databricks ecosystem.
• Drive continuous improvement and innovation in platform operations, monitoring, and performance.
• Collaborate with data engineers, data scientists, governance, and operations teams to deliver automated solutions that enhance productivity, platform stability and improve customer experience.

Required Skills & Qualifications :

Experience :

AWS (Mandatory) :

DevOps and Automation (Mandatory) :

Nice to have :

Big Data & Data Platforms :