Talent.com
Sr Data Engineer with GCP, Spark, Python
Sr Data Engineer with GCP, Spark, PythonInterSources • Sunnyvale, CA, United States
No longer accepting applications
Sr Data Engineer with GCP, Spark, Python

Sr Data Engineer with GCP, Spark, Python

InterSources • Sunnyvale, CA, United States
16 days ago
Job type
  • Full-time
Job description

Sr Data Engineer with GCP, Spark, Python

Sunnyvale, CA (Onsite)

Long Term

Requirements : GCP Experience

  • 2+ years of recent GCP experience
  • Experience building data pipelines in GCP
  • GCP Dataproc, GCS & BIGQuery experience
  • 5+ years of hands-on experience with developing data warehouse solutions and data products.
  • 5+ years of hands-on experience developing a distributed data processing platform with Hadoop, Hive or Spark, Airflow or a workflow orchestration solution are required
  • 2+ years of hands-on experience in modeling and designing schema for data lakes or for RDBMS platforms.
  • Experience with programming languages : Python, Java, Scala, etc.
  • Experience with scripting languages : Perl, Shell, etc.
  • Practice working with, processing, and managing large data sets (multi TB / PB scale).
  • Exposure to test driven development and automated testing frameworks.
  • Background in Scrum / Agile development methodologies.
  • Capable of delivering on multiple competing priorities with little supervision.
  • Excellent verbal and written communication skills.
  • Bachelor's Degree in computer science or equivalent experience.

The most successful candidates will also have experience in the following :

  • Gitflow
  • Atlassian products - BitBucket, JIRA, Confluence etc.
  • Continuous Integration tools such as Bamboo, Jenkins, or TFS
  • Responsibilities :

  • Design and develop big data applications using the latest open source technologies.
  • Desired working in offshore model and Managed outcome
  • Develop logical and physical data models for big data platforms.
  • Automate workflows using Apache Airflow.
  • Create data pipelines using Apache Hive, Apache Spark, Apache Kafka.
  • Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support.
  • Learn our business domain and technology infrastructure quickly and share your knowledge freely and actively with others in the team.
  • Mentor junior engineers on the team
  • Lead daily standups and design reviews
  • Groom and prioritize backlog using JIRA
  • Act as the point of contact for your assigned business domain
  • About Us :

    InterSources Inc , a Certified Diverse Supplier, was founded in 2007 and offers innovative solutions to help clients with Digital Transformations across various domains and industries. Our history spans over 16 years and today we are an Award-Winning Global Software Consultancy solving complex problems with technology. We recognize that our employees and our clients are our strengths as the diverse talents and opportunities they bring to the table enable us to grow as a global platform and they are causally linked with our success. We provide strategic and technical advice, and we have expertise in areas covering Artificial Intelligence, Cloud Migration, Custom Software Development, Data Analytics Infrastructure & Cloud Solutions, Cyber Security Services, etc. We make reasonable accommodations for clients and employees and we do not discriminate based on any protected attribute including race, religion, color, national origin, gender sexual orientation, gender identity, age, or marital status. We also are a Google Cloud partner company. We align strategy with execution and provide secure service solutions by developing and using the latest technologies that thrive our resources to deliver industry-leading capabilities to our clients and customers, making it convenient for our clients to do business with InterSources Inc. Our teams also drive growth by refining technology-driven client experiences that put the users first, providing an unparalleled experience. This results in strengthening the core technologies of clients, enabling them to scale with flexibility, create seamless digital experiences and build lifelong relationships.

    Create a job alert for this search

    Sr Data Engineer • Sunnyvale, CA, United States

    Related jobs
    Sr. Data Engineer

    Sr. Data Engineer

    Contact Government Services, LLC • San Francisco, CA, US
    Full-time
    Employment Type : Full-Time, Mid-level.Department : Business Intelligence.CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence pl...Show more
    Last updated: 7 hours ago • Promoted • New!
    Sr. / Staff Software Engineer, Big Data

    Sr. / Staff Software Engineer, Big Data

    Predactiv • Palo Alto, CA, US
    Full-time
    ShareThis, a Predactiv Company is a big data company that owns online behavior data of 1b+ users globally.We are developing an audience intelligence platform with cutting edge big data technologies...Show more
    Last updated: 30+ days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    FocusKPI Inc. • Mountain View, CA, US
    Temporary
    FocusKPI is seeking a .The team is seeking world-class server software engineers with experience in Big Data Infrastructure and Data warehousing to join their technology innovation group,...Show more
    Last updated: 5 days ago • Promoted
    Senior Software Engineer, Big Data

    Senior Software Engineer, Big Data

    ZipRecruiter • Palo Alto, CA, US
    Full-time
    We offer a hybrid work environment.Most US-based positions can also.To actively connect people to their next great opportunity. ZipRecruiter is a leading online employment marketplace.Powered by AI-...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data & Analytics Engineer

    Senior Data & Analytics Engineer

    Meshy • Sunnyvale, CA, US
    Full-time
    Headquartered in the Silicon Valley, Meshy is the leading 3D generative AI company on a mission to.Meshy makes it effortless for both professional artists and hobbyists to create unique 3D assets&m...Show more
    Last updated: 4 days ago • Promoted
    AI Incubator - Data Engineer

    AI Incubator - Data Engineer

    Sprinter Health • Menlo Park, CA, US
    Full-time
    At Sprinter Health, our mission is reimagining how people access care by bringing it directly to their homes.Nearly 30% of patients in the U. For many, the ER becomes their first touchpoint with the...Show more
    Last updated: 7 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    AngelList • San Francisco, CA, US
    Full-time
    We exist to accelerate innovation.We do this by giving more people the opportunity to participate in the venture economy by building the financial infrastructure that makes it possible for more peo...Show more
    Last updated: 7 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    Polly • San Francisco, CA, US
    Full-time
    You’re a hands on Senior Data Engineer with strong Python skills and experience building robust data pipelines in cloud environments. You enjoy solving complex data problems, working cross-fun...Show more
    Last updated: 7 hours ago • Promoted • New!
    Sr. Staff Engineer, AI and Data Platform

    Sr. Staff Engineer, AI and Data Platform

    Quizlet • San Francisco, CA, US
    Full-time
    At Quizlet, our mission is to help every learner achieve their outcomes in the most effective and delightful way.Our $1B+ learning platform serves tens of millions of students every month, in...Show more
    Last updated: 7 hours ago • Promoted • New!
    Data Engineer

    Data Engineer

    Forhyre • Sunnyvale, CA, US
    Full-time
    We are looking for a passionate certified Data Engineer.The successful candidate will turn data into information, information into insight and insight into business decisions.Data analyst responsib...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Plum Inc • San Francisco, CA, US
    Full-time
    PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    SteerBridge • Miramar, CA, US
    Full-time
    SteerBridge Strategies is a CVE-Verified Service-Disabled, Veteran-Owned Small Business (SDVOSB) delivering a broad spectrum of professional services to the U. Backed by decades of hands-on experien...Show more
    Last updated: 7 hours ago • Promoted • New!
    Sr. Software Engineer - Data Engineer

    Sr. Software Engineer - Data Engineer

    FyrFly Venture Partners • Pleasanton, CA, United States
    Full-time
    Software Engineer - Data Engineer.Senior Software Engineer – Data Engineer.This role will focus on building and optimizing data pipelines, integrating multiple data sources, and enabling advanced a...Show more
    Last updated: 15 days ago • Promoted
    Sr. Data Engineer

    Sr. Data Engineer

    Machinify • Palo Alto, CA, US
    Full-time
    Machinify is a leading healthcare intelligence company with expertise across the payment continuum, delivering unmatched value, transparency, and efficiency to health plan clients across the countr...Show more
    Last updated: 30+ days ago • Promoted
    Data Analytics Engineer

    Data Analytics Engineer

    Parafin • San Francisco, CA, US
    Full-time
    At Parafin, we’re on a mission to grow small businesses.Small businesses are the backbone of our economy, but traditional banks often don’t have their backs. We build tech that makes it ...Show more
    Last updated: 30+ days ago • Promoted
    Sr Staff Engineer Software (Prisma Access Data-Plane Applications)

    Sr Staff Engineer Software (Prisma Access Data-Plane Applications)

    Palo Alto Networks • Santa Clara, CA, US
    Full-time
    At Palo Alto Networks® everything starts and ends with our mission : .Being the cybersecurity partner of choice, protecting our digital way of life. Our vision is a world where each day is safer a...Show more
    Last updated: 30+ days ago • Promoted
    Data Analysis Engineer (Python / Tableau ON-SITE) (Cupertino)

    Data Analysis Engineer (Python / Tableau ON-SITE) (Cupertino)

    OSI Engineering • Cupertino, CA, US
    Part-time
    Seeking a Data Analysis Engineer.The role involves building Tableau dashboards, Python scripting, and working with hardware-generated data. This role will focus on building interactive Tableau dashb...Show more
    Last updated: 2 days ago • Promoted
    Snowflake Data Engineer

    Snowflake Data Engineer

    Axiom Software Solutions Limited • Sunnyvale, CA, US
    Full-time
    Strong expertise coding in Snowflake and Python with 5+ years of experience.Hands on knowledge of Spark and PySpark is an added advantage. Ability to collaborate with business users to gather requir...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer 4

    Data Engineer 4

    NextDeavor Inc. • San Jose, CA, US
    Temporary
    Here's how you'll become a key player with this opportunity : .We are looking for a Splunk expert to join on a short-term contract and help stabilize, optimize, and improve our Splunk environ...Show more
    Last updated: 30+ days ago • Promoted
    Sr. Data Engineer

    Sr. Data Engineer

    Visa • Foster City, CA, United States
    Full-time
    Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more t...Show more
    Last updated: 1 day ago • Promoted