Principal Data Processing Engineer

Principal Data Processing Engineer

Mountain View, CA

About DataPelago :

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.

The Opportunity :

As a Principal Data Processing Engineer, you will be a key technical leader of core execution engine components of our data processing engine. You will lead architecture, design, and implementation that will enhance functional breadth, performance, scale, and reliability of the engine to deliver a product that will redefine how users extract intelligence from their data. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.

What You'll Do :

• Architectural Leadership : Drive the evolution of our parallel and distributed execution en-

gine architecture, with a strong focus on leveraging accelerated computing technologies.

implementation, and rollout of an enterprise-grade product.

of the data processing execution engine.

identify opportunities for the engine to enhance technology and product leadership.

and customer success teams. Guide and mentor engineers on the execution engine team.

and issue resolution to maintain highest product quality, security, efficiency, & productivity.

What You'll Bring :

database or analytics execution engines designed for large-scale data processing.

Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion, or

Apache DataFusion Comet is highly preferred.

successful release of high-performance data processing engines for large production de-

ployments.

technical concepts to both technical and non-technical audiences.

Location Considerations :

We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and atremote locations. This specific position will be based at our headquarters in Mountain View, CA.

Why Join DataPelago?

our core parallel analytics engine.

computing and data processing.

of our mission-critical platform.

own technical expertise.

velopment opportunities.