Senior Software Engineer

Optomi, in partner with a leader in entertainment, is looking for a Software Engineer with strong software engineering fundamentals to design, build, and generalize data transformation pipelines. This role sits at the intersection of data engineering and platform development - ideal for an engineer who’s equally comfortable with large-scale data systems and modern software engineering practices. You’ll develop scalable, production-grade data solutions that allow the marketing analytics ecosystem to target new personas and integrate with additional platforms.

Job Description

Responsibilities of the right candidate :

• Design, build, and optimize streaming and batch data pipelines using PySpark (preferably PySpark Streaming)
• Develop a data integration platform in Python to process high-volume event data
• Generalize and modularize existing pipelines for broader persona targeting and partner integrations
• Collaborate with marketing, product, and engineering teams to design new data-driven features
• Implement CI / CD workflows with GitHub Actions and write comprehensive unit tests
• Leverage Airflow for orchestration and scheduling of complex data workflows
• Containerize and deploy applications using Docker (and possibly Kubernetes)
• Maintain and scale production data systems running in cloud or EMR environments (Databricks or equivalent)

Experience of the right candidate :