Talent.com
Data Engineer -(GCP,Airflow,Pyspark) Hybrid-Sunnyvale,CA
Data Engineer -(GCP,Airflow,Pyspark) Hybrid-Sunnyvale,CAKaav Inc. • Sunnyvale, CA, United States
Data Engineer -(GCP,Airflow,Pyspark) Hybrid-Sunnyvale,CA

Data Engineer -(GCP,Airflow,Pyspark) Hybrid-Sunnyvale,CA

Kaav Inc. • Sunnyvale, CA, United States
3 days ago
Job type
  • Full-time
Job description

Description :

  • Bachelor or master's degree in computer science, Software Engineering, or a related field.
  • Proven experience (7+ years) as a data Engineer, preferably with a focus on software development, distributed systems.
  • Solid understanding of data engineering concepts, database design, ETL processes and data mining.
  • Proficiency in working with data technologies including SQL, Python, Spark, Scala, Hadoop and related technologies and tools.
  • Experience with ETL tools such as Apache Airflow, digdag, oozie.
  • Experience with?CI / CD?processes and tools such as Jenkins and Maven?
  • Strong understanding of metadata management, data lineage and data quality and tools.
  • Familiarity with Cloud Computing Services such as Google GCP and Microsoft Azure, as well as Distributed Storage Systems like Hive and Elastic Search a plus.
  • Experience implementing operational best practices such as monitoring, alerting, validation and exception handling.
  • Strong analytical, problem-solving skills with the ability to optimize, troubleshoot and debug data process and pipelines.
  • Excellent communication skills to collaborate effectively with cross-functional teams and present performance analysis findings.
  • Experience supporting and working with cross-functional teams and influence them with solutions in a dynamic environment.
  • Experience in?AdTech and advertising measurement
  • Knowledge of analytical tools such as Tableau / or Looker is preferred.

What project or initiative will they be working on?

  • Universal Onboarding / Account.
  • Will this role be hybrid?

  • Yes
  • If hybrid, how many days per week will need to be in office?

  • 2.
  • Top 3 Skills Needed or Required?

  • GCP,Airflow,Pyspark,Tableau.
  • What are the day-to-day responsibilities?

  • Build and maintain robust, scalable data pipelines for ingesting, transforming, and storing large volume of data to support client connect and its advertising needs.
  • Work on cloud platforms like Azure and Google Cloud for data storage, processing and analytics.
  • Create and deploy large-scale, containerized applications using Docker and Kubernetes in public clouds like Google GCP and Microsoft Azure.
  • Create and deploy large-scale, containerized applications using Docker and Kubernetes in public clouds like Google GCP and Microsoft Azure.
  • Manage data engineering projects from design to deployment, ensuring timely delivery and meeting project goals.
  • Self-starter, mentor who can lead engineers and engineering projects.
  • Coordinate, create, and complete technical design discussions to drive technical architecture.
  • Navigate complex systems?and deliver highly scalable and reliable production-ready code?
  • Understands, articulates, and applies principles of the defined strategy to routine business problems that involve a single function.
  • Support data operations including on-call rotations to troubleshoot production issues partnering with cross-functional teams.
  • Collaborate with cross-functional tech teams including developers, architects, operations to identify performance bottlenecks, system issues, and areas for optimization.
  • What is the makeup of the team?

  • Currently ABS-Data team consists of 10 associates who work on delivering various initiatives to support client Connect's roadmap.
  • Additional Job Details

  • GCP,Airflow,Pyspark,Tableau.
  • Required Skills : Data Analysis

    Additional Skills : Data EngineerThis is a high PRIORITY requisition. This is a PROACTIVE requisition

    Create a job alert for this search

    Data Engineer • Sunnyvale, CA, United States

    Related jobs
    Senior GCP Data Engineer

    Senior GCP Data Engineer

    Jconnect Inc • Sunnyvale, CA, United States
    Full-time
    Below is the requirement with my client.Please let me know if you are available for this role.Title : Senior GCP Data Engineer. Hadoop, Hive or Spark, Airflow or a workflow orchestration solution are...Show more
    Last updated: 3 days ago • Promoted
    Senior Data Engineer (AI / ML, Cloud) - Full Time - Bay Area, CA

    Senior Data Engineer (AI / ML, Cloud) - Full Time - Bay Area, CA

    Right Skale, Inc. • Pleasanton, CA, United States
    Full-time
    Senior Data Engineer (AI / ML, Cloud) - Full Time - Bay Area, CA.We are seeking a Senior Data Engineer to design and implement AI / ML-driven data solutions for enterprise cloud environments.You'll lev...Show more
    Last updated: 13 days ago • Promoted
    Technical Lead (GCP Data Engineer)

    Technical Lead (GCP Data Engineer)

    Tekfortune Inc • San Jose, CA, United States
    Permanent +1
    Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries.In this quickly ch...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer GCP

    Data Engineer GCP

    Interaction24 LLC • Sunnyvale, CA, United States
    Full-time
    About the job Data Engineer GCP.GoogleAds Expertise is a plus (campaign / adsgroups setup APIs).Show more
    Last updated: 30+ days ago • Promoted
    GCP - Data Engineer

    GCP - Data Engineer

    Omni Inclusive • San Jose, CA, United States
    Full-time
    Strong knowledge in writing and interpreting complex SQL queries - Experience in GCP.Strong ability to design, develop, test and maintain ETL solution in Google Cloud Platform especially in BigQuey...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer - Hybrid

    Data Engineer - Hybrid

    The Dignify Solutions LLC • Sunnyvale, CA, United States
    Full-time
    Python or Scala or Java and experience optimizing SQL queries on large data.Experience working with large-scale data warehouse solutions such as Teradata, Snowflake, or Redshift.Hands on experience...Show more
    Last updated: 2 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Texas State Library and Archives Commision • Sunnyvale, CA, United States
    Full-time
    Sample profile attached for your reference.Recent update from client for this data engineer role / Request-ID : 13145-1 _ Bentonville, AR position is no longer available. We can submit profiles for S...Show more
    Last updated: 3 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    ApTask • Sunnyvale, CA, United States
    Full-time
    Job Description : Senior Data Engineer.Location : Sunnyvale, CA (Local candidates preferred).Key Skills : Tableau, Python, Snowflake. The ideal candidate will excel at collaborating with business users...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    Adobe • San Jose, CA, United States
    Full-time
    Changing the world through digital experiences is what Adobe's all about.We give everyone-from emerging artists to global brands-everything they need to design and deliver exceptional digital exper...Show more
    Last updated: 30+ days ago • Promoted
    GCP Data Engineer

    GCP Data Engineer

    Cynet Systems • San Jose, CA, United States
    Full-time
    Require 5-7 + years of experience Data Engineers GCP Strong knowledge in writing and interpreting complex SQL queries Experience in GCP is plusStrong ability to design, develop, test and maintain E...Show more
    Last updated: 3 days ago • Promoted
    Data Pipeline Engineer IV

    Data Pipeline Engineer IV

    Pyramid Consulting • Mountain View, CA, United States
    Temporary
    Please review the job description below and contact me ASAP if you are interested.Employee benefits include, but are not limited to, health insurance (medical, dental, vision), 401(k) plan, and pai...Show more
    Last updated: 3 days ago • Promoted
    Snowflake Data Engineer

    Snowflake Data Engineer

    Diverse Lynx • Sunnyvale, CA, United States
    Full-time
    Lead the design and implementation of scalable data ingestion architectures and pipelines into Snowflake.Develop, maintain, and optimize ETL / ELT processes using Snowflake features like Snowpipe, St...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer - AG

    Data Engineer - AG

    Bayone • Sunnyvale, CA, United States
    Full-time
    Demonstrates up-to-date expertise and applies this to the development,.Supporting and aligning efforts to meet customer, business. Create software design and architecture for next software solution....Show more
    Last updated: 30+ days ago • Promoted
    Data Pipeline Engineer IV

    Data Pipeline Engineer IV

    Hire Talent • Mountain View, CA, United States
    Full-time
    Data Engineer - Mobile Cloud ServiceLocation : Mountain View, CAGeneral Description : The Cloud services group is looking for world class engineers to join our technology innovation group focused on t...Show more
    Last updated: 3 days ago • Promoted
    Senior Data Engineer (only local to Sunnyvale, CA)

    Senior Data Engineer (only local to Sunnyvale, CA)

    Inficare • Sunnyvale, CA, United States
    Full-time
    Job Title : Senior Data Engineer (Tableau, Python, Snowflake).Location : Sunnyvale, CA ( 3 days Hybrid).The ideal candidate will excel at collaborating with business users, cross-functional teams, an...Show more
    Last updated: 3 days ago • Promoted
    Data Engineer with GCP

    Data Engineer with GCP

    HCL Global Systems • Sunnyvale, CA, United States
    Full-time
    Sr : Data Engineer Description : We are looking for a highly energetic and collaborative Data Engineer for a 12-month engagement. Responsibilities : As a Senior Data Engineer, you will.Design and devel...Show more
    Last updated: 3 days ago • Promoted
    Data Engineer

    Data Engineer

    Akkodis • Sunnyvale, CA, United States
    Full-time
    This role requires advanced expertise in Snowflake, SQL, and Python to build scalable data pipelines and implement dimensional data models. The rate may be negotiable based on experience, education,...Show more
    Last updated: 3 days ago • Promoted
    Senior Data Engineer (Spark, python, AWS)-US

    Senior Data Engineer (Spark, python, AWS)-US

    Zortech Solutions • Mountain View, CA, United States
    Full-time
    Role : Senior Data Engineer (Spark, Python, AWS).Location : Mountain View California (100% Onsite).Spark, Python, SQL, DWH / ETL Concepts, AWS. Design, build and maintain data infrastructure that supp...Show more
    Last updated: 2 days ago • Promoted