As a Principal Data Engineer, you will lead a team in building and maintaining scalable, reliable data pipelines that bridge on-premises and cloud environments. You'll leverage your expertise in data engineering, streaming technologies, and leadership to drive the team towards achieving business objectives.
Key Responsibilities:
Team Leadership: Lead, mentor, and manage a team of data engineers specializing in streaming technologies.
Data Pipeline Development: Design and implement high-throughput, low-latency streaming data pipelines using Apache Kafka, ensuring integration with cloud services (e.g., BigQuery, Looker).
Data Analytics: Oversee the development of stream processing applications using Apache Spark or Apache Flink, and implement real-time data transformations and analytics using KSQL.
Data Storage: Design and maintain scalable data storage solutions with ClickHouse for fast analytics on streaming data.
ETL Processes: Lead the design and implementation of ETL processes, extracting, transforming, and loading data into a data warehouse.
Data Quality: Ensure data integrity, consistency, and accuracy through robust data quality assurance.
Optimization: Optimize performance and implement best practices in data engineering, covering data quality, security, and efficiency.
Collaboration: Collaborate with stakeholders to gather requirements and align data strategies with business objectives.
Technology Updates: Stay current with emerging technologies in streaming and cloud environments, evaluating their potential application.
Qualifications:
5+ years of hands-on data engineering experience with Python, Scala, or Java
3+ years of experience with cloud vendors (AWS, Azure, GCP), data warehouse services (e.g., Redshift, Databricks), and cloud storage
Expertise in KSQL, ClickHouse, ETL/ELT tools (e.g., Airflow, ADF, Glue, NiFi), and orchestration
Proficiency in code versioning (Git) and CI/CD pipelines
Experience with stream processing tools (Apache Kafka, Apache Flink, Apache Spark Structured Streaming)
Strong understanding of data modeling, optimization techniques, and stream processing patterns
Excellent leadership and mentorship skills with at least 2 years in a leadership role
Nice to Have:
Experience with NoSQL databases, Kafka Connect, Kafka Streams, Superset for data visualization
Knowledge of integrating real-time machine learning models into streaming environments
Expertise with monitoring and observability tools for streaming systems
Why Join Overwatch Agriculture?
At Overwatch Agriculture, we prioritize people over processes, fostering a supportive and tech-savvy environment. Our customized benefits prioritize your well-being and professional growth, making this an ideal opportunity for tech enthusiasts.
Principal Data Engineer • Boulder, Colorado, United States
Known for being a great place to work and build a career, KPMG provides audit, tax and advisory services for organizations in today's most important industries.Our growth is driven by delivering re... Show more
When it comes to finding the perfect locums assignment, sometimes it is all about who you know.CompHealth has been around for a long time and have a vast network of connections to facilities across... Show more
A leading space analytics company in Colorado is seeking a Spacecraft Software Engineer (GNSS) to develop mission-critical software for satellite payloads.Candidates should have expertise in C++ or... Show more
SciTec, a wholly owned subsidiary of Firefly Aerospace, is a dynamic non-traditional defense contractor that delivers advanced technologies in support of U.For the past forty-five plus years, we ha... Show more
At LifeStance Health, we believe in a truly healthy society where mental and physical healthcare are unified to make lives better.Our mission is to help people lead healthier, more fulfilling lives... Show more
Join Our Team as a Territory Manager – Where Passion Meets Opportunity!.Are you ready to build relationships, drive sales, and make a meaningful impact in the foodservice industry? At US Foods, we’... Show more
SciTec has been awarded multiple government contracts and is growing our creative Team! SciTec, Inc.We support customers throughout the Department of Defense and U.Government in building innovative... Show more
The Agent Platform team delivers a new AI‑focused experience for Workday customers and their employees.We are at the forefront of Workday’s AI & Agentic initiatives, delivering and integrating Agen... Show more
Work as part of a highly skilled team to help develop, maintain, and support print related products on the mainframe platform.Assist with coding, testing, and debugging tasks while learning establi... Show more
Boulder Community Health, Boulder, CO 80303.Monday-Friday 8am-4pm with an on-call requirement.Colorado residency required; regular onsite attendance expected.Boulder Community Health is seeking an ... Show more
Otter's software engineers develop next-generation technologies for the outdoor industry that change how billions of users sign waivers, explore, and interact with one another.Our products need to ... Show more
Outside Analytics has recently become a proud subsidiary of SMX, marking an exciting collaboration that enhances our collective capabilities to deliver cutting-edge digital transformation solutions... Show more
Cloud DevOps Engineer Location: Denver, CO or Boulder, CO Work Mode: Hybrid, 2 days onsite and 3 remote per week Employment Type: Full-time, Direct Hire Salary: $95k - $130k (depending on experienc... Show more
Creating Peace of Mind by Pioneering Safety and Security.At Allegion, we help keep the people you know and love safe and secure where they live, work and visit.With more than 30 brands, 12,000+ emp... Show more
As an integral member of our platform team, you bring both depth and experience to building enterprise-grade software.We are in the process of implementing a service-oriented approach to our produc... Show more
SciTech is a dynamic small business, with the mission to deliver advanced sensor data processing technologies and scientific instrumentation capabilities in support of National Security and Defense... Show more
Real-world expertise is turned into training data, evaluations, and feedback loops that improve how models perform.AI labs and enterprises use micro1 to train models and build reliable AI agents th... Show more
We're transforming one of the world’s oldest industries with cutting‑edge technology and an innovative approach.Backed by top‑tier investors and recognized by Time as one of the “best Inventions of... Show more
A technology company specializing in aviation solutions seeks a Senior Software Engineer to lead the development of core product features and optimize data transformation pipelines.Located in Bould... Show more
The application window is expected to close on: 04/29/2026.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.Develops software cons... Show more
