Senior Data Engineer

United IT SolutionsMountain View, CA, United States

5 hours ago

Job type

Full-time

Job description

Role : Senior Data Engineer - GCP

Location : Mountain View CA (Onsite)

What you'll do

Develop and enhance Python frameworks and libraries to support data processing, quality, lineage, governance, analysis, and machine learning operations.
Design, build, and maintain scalable and efficient data pipelines on GCP.
Implement robust monitoring, logging, and alerting systems to ensure the reliability and stability of data infrastructure.
Build scalable batch pipelines leveraging Big query, Dataflow and Airflow / Composer scheduler / executor framework on Google Cloud Platform
Building data pipelines, leveraging Scala, Pub Sub, Akka, Dataflow on Google Cloud Platform
Design our data models for optimal storage and retrieval and to meet machine learning modeling using technologies like Bigtable and Vertex Feature Store
Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company

Minimum Basic Requirements

Python Expertise : Write and maintain Python frameworks and libraries to support data processing and integration tasks.

Code Management : Use Git and GitHub for source control, code reviews, and version management.

GCP Proficiency : Extensive experience working with GCP services (e.g., BigQuery, Cloud Dataflow, Pub / Sub, Cloud Storage).

Python Mastery : Proficient in Python with experience in writing, maintaining, and optimizing data processing frameworks and libraries.

Software Engineering : Strong understanding of software engineering best practices, including version control (Git), collaborative development (GitHub), code reviews, and CI / CD.

Data Management : Deep knowledge of data modeling, ETL / ELT, and data warehousing concepts.

Problem-Solving : Excellent problem-solving skills with the ability to tackle complex data engineering challenges.

Communication : Strong communication skills, including the ability to explain complex technical details to non-technical stakeholders.

Data Science Stack : Proficiency in data analysis and familiarity with tools such as Jupyter Notebook, pandas, NumPy, and other Python data analysis libraries.

Frameworks / Tools : Familiarity with machine learning and data processing tools and frameworks such as TensorFlow, Apache Spark, and scikit-learn.

Bachelor's or master's degree in computer science, Engineering, Computer Information Systems, Mathematics, Physics, or a related field or software development training program

Preferred Qualifications

Experience in Scala, Java, and / or any functional language. We code primarily in Scala, so you'll be excited to either ramp or continue with such

Experience in microservices architecture, messaging patterns, and deployment models

Experience in API design and building robust and extendable client / server contracts

Create a job alert for this search

Senior Data Engineer • Mountain View, CA, United States

Related jobs

Promoted

Senior Data Engineer

KiddomSan Francisco, CA, United States

Full-time +1

Kiddom is redefining how technology powers learning.We combine world-class curriculum with cutting-edge AI and modern SaaS infrastructure to help schools deliver truly personalized education at sca...Show moreLast updated: 30+ days ago

Promoted
New!

Senior Data Engineer

Tekfortune IncSunnyvale, CA, United States

Permanent

Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries.In this quickly ch...Show moreLast updated: 5 hours ago

Promoted

Senior Data Engineer

FigureSan Jose, CA, United States

Full-time

Figure is an AI Robotics company developing a general-purpose humanoid.Our humanoid is designed for corporate tasks targeting labor shortages and jobs that are undesirable or unsafe.We are based in...Show moreLast updated: 30+ days ago

Promoted
New!

Senior / Staff Data Engineer

OKXSan Jose, CA, United States

Full-time

At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom.OKX is a leading crypto exchange, and the developer of OKX Wallet, giving mill...Show moreLast updated: 3 hours ago

Promoted

Senior Data Engineer

The Clorox CompanyPleasanton, CA, United States

Full-time

Clorox is the place that's committed to growth - for our people and our brands.Guided by our purpose and values, and with people at the center of everything we do, we believe every one of us can ma...Show moreLast updated: 13 days ago

Promoted
New!

Senior Data Engineer

Texas State Library and Archives CommisionSunnyvale, CA, United States

Full-time

Sample profile attached for your reference.Recent update from client for this data engineer role / Request-ID : 13145-1 _ Bentonville, AR position is no longer available. We can submit profiles for S...Show moreLast updated: 5 hours ago

Promoted

Senior Data Engineer

TWO95 InternationalSunnyvale, CA, United States

Full-time

Provide business insights, while leveraging internal tools and systems, databases and industry data.Ability to document requirements, data lineage, subject matter in both business and technical ter...Show moreLast updated: 30+ days ago

Promoted

Senior Analytics Engineer

Color HealthBurlingame, CA, United States

Full-time

Color Health is revolutionizing cancer care with the nation's first Virtual Cancer Clinic, delivering high-quality, physician-led multidisciplinary care across all 50 states.Our innovative, guideli...Show moreLast updated: 18 days ago

Promoted
New!

Senior Data Engineer

ApTaskMountain View, CA, United States

Full-time

The client provides information technology (IT) services, including business outsourcing, infrastructure technology, and application services. The application service offered by the company includes...Show moreLast updated: 3 hours ago

Promoted

Senior Data Engineer

AthenaPalo Alto, CA, United States

Full-time

At Athena, we empower possibility through transformative delegation.True leaders reflect on what they want and map the path to get there. We clear the way by pairing exceptional Executive Assistants...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer, SWE Operations & Planning

AppleCupertino, CA, United States

Full-time

Apple is where individual imaginations gather together, committing to the values that lead to great work.Every new product we build, service we create, or Apple Store experience we deliver is the r...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

LiveRampSan Francisco, CA, United States

Full-time

LiveRamp is the data collaboration platform of choice for the world's most innovative companies.A groundbreaking leader in consumer privacy, data ethics, and foundational identity, LiveRamp is sett...Show moreLast updated: 1 day ago

Promoted

Senior Data Engineer

iO AssociatesSan Francisco, CA, United States

Full-time

Job Title : Senior Data Engineer.Our Client is a leading company in the technology industry, dedicated to innovation, growth, and creating a dynamic work culture. With a strong reputation for excelle...Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer

Info Way SolutionsMountain View, CA, United States

Full-time

Job Title : Senior Data Engineer (High Priority).Location : Preferably Mountain View, CA; Hybrid.This role leans heavily into core data engineering - from pipeline development to data modeling - and ...Show moreLast updated: 30+ days ago

Promoted
New!

Senior Data Engineer

Jconnect IncSunnyvale, CA, United States

Full-time

Below is the requirement with my client.Please let me know if you are available for this role.Location : Sunnyvale CA (Onsite). Experience building data pipelines in GCP.GCP Dataproc, GCS & BIGQuery ...Show moreLast updated: 5 hours ago

Promoted

Senior Data Engineer

Toyota Research InstituteLos Altos, CA, United States

Full-time

At Toyota Research Institute (TRI), we're on a mission to improve the quality of human life.We're developing new tools and capabilities to amplify the human experience. To lead this transformative s...Show moreLast updated: 30+ days ago

Promoted

Senior Data Platform Engineer

Crustdata (YC F24)San Francisco, CA, United States

Full-time

This range is provided by Crustdata (YC F24).Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. We are looking for a foundational member of our eng...Show moreLast updated: 4 days ago

Promoted

Senior AI & Data Platform Engineer

Icon VenturesSan Francisco, CA, United States

Full-time

At Quizlet, our mission is to help every learner achieve their outcomes in the most effective and delightful way.Our $1B+ learning platform serves tens of millions of students every month, includin...Show moreLast updated: 2 days ago