Data and Streaming Engineer

Job Title : Senior Data & Streaming Engineer

Location : Louisville, KY

Job Overview

We are seeking a highly skilled Senior Data & Streaming Engineer to join our team on a contract basis. The ideal candidate will bring deep expertise in Databricks, Apache Spark, Delta Lake, real-time streaming technologies (Kafka / EventHub / Kinesis), SQL , and modern orchestration tools such as Airflow, dbx, and Databricks Workflows .

This role focuses on designing, building, and operationalizing high-performance batch and real-time data pipelines that support mission-critical analytics, machine learning feature generation, and low-latency decisioning systems. The engineer will work across AWS / Azure cloud platforms and contribute to production-grade, scalable, observable pipelines.

Key Responsibilities

• Design and develop robust batch and streaming pipelines to support real-time ingestion, stateful processing, feature computation, and enrichment with exactly-once semantics .
• Build and maintain application-ready, low-latency datasets and optimized data transformation layers.
• Integrate, enhance, and operate machine learning feature pipelines and real-time signals supporting the NBA decision engine.
• Monitor, troubleshoot, and tune data pipelines for performance, reliability, and scalability , ensuring adherence to strict SLAs.
• Implement proactive alerting, job orchestration, root-cause analysis , and create operational dashboards.
• Deliver well-documented, observable, and independently manageable pipelines, dashboards, runbooks , and incident-resolution processes.
• Utilize GenAI coding assistants (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) to accelerate development, diagnostics, automation, and documentation.

Required Skills & Experience

Preferred Qualifications

Background in supporting decision engines or real-time analytical workloads

eye