Talent.com
Cloud Data Engineer
Cloud Data EngineerGovx • New York, New York, United States
Cloud Data Engineer

Cloud Data Engineer

Govx • New York, New York, United States
19 hours ago
Job type
  • Full-time
  • Remote
Job description

POSITION DESCRIPTION

The Cloud Data Engineer is part of a Data team that is responsible for supporting, modernizing, and transforming our data and reporting capabilities across our products by implementing a new modernized data architecture. The position will be responsible for day-to-day data collection, transportation, maintenance / curation, and access to the GOVX corporate data assets. The Cloud Data Engineer will work cross-functionally across the enterprise to centralize data and standardize it for use by business reporting, machine learning, AI, data science, and other stakeholders. This position plays a critical role in increasing awareness about available data and democratizing access to it across GOVX and our data partners.

This position will be under the supervision of the Business Intelligence Manager.

RESPONSIBLITIES

  • Supporting and modernizing existing data integrations.
  • Crafting and maintaining efficient data pipeline architecture.
  • Assembling large, complex data sets that meet business requirements.
  • Create and maintain optimal data pipeline / flow architecture.
  • Identifying, crafting, and implementing internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Partner with business analysts, data scientists, and IT teams to translate business requirements into scalable data solutions using Fabric, Spark, using Delta Lake. Develop Spark notebooks for both batch and streaming ETL pipelines leveraging Delta Lake.
  • Implement and optimize Delta Lake features including schema enforcement, schema evolution, and time travel for robust data management. Optimize Delta Lake tables for performance using Z-ordering, compaction, and partitioning strategies.
  • Working with the team to strive for clean and meaningful data, and greater functionality and flexibility within the team’s data systems.
  • Design processes supporting data transformation, data structures, metadata, dependency, and workload management.

SKILL / REQUIREMENTS

  • Hands-on experience developing, debugging, and optimizing Spark notebooks for ETL and analytics in Microsoft Fabric and Azure.
  • Deep expertise in Microsoft Fabric, Dataflows Gen2, and Power BI integration.
  • Hands-on experience with Delta Lake table management, including schema evolution, versioning, and data compaction
  • Experience with Data Lakehouse and Medallion Architecture.
  • Experience with CI / CD and version control using Git.
  • Advanced SQL and NoSQL query authoring; Python and Spark scripting.
  • Proficiency with object-oriented / object function scripting languages : Python, Spark, etc.
  • Proficiency with Metadata-Driven Design and JSON.
  • Experience working with streams such as Event Hubs and Event Driven Architectures.
  • Experience building, maintaining, and optimizing ‘big data’ data pipelines, architectures, and data sets.
  • Experience cleaning, testing, and evaluating data quality from a wide variety of ingestible data sources.
  • Knowledge Microsoft Power Platform including Copilot Studio and Power Apps.
  • Strong collaboration and communication skills with business and technical teams.
  • SUPERVISORY RESPONSIBILITY

    This position has no supervisory responsibilities.

    WORK ENVIRONMENT

    This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines. This role occasionally must lift and carry the office equipment. Occasional evening, night and weekend shifts are required.

    Work Location

    Due to state law and tax implications, remote work candidates must live and work in one of the following states : California, Texas, Washington, Florida, Tennessee, New York, or Colorado.

    PHYSICAL / MENTAL DEMANDS

  • Physical - This is a sedentary role.
  • Mental – Problem solving, making decisions, interpreting data, organize, read / write.
  • Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.
  • TRAVEL

    Annual travel to the San Diego office headquarters is expected for this position.

    PREFERRED EDUCATION AND EXPERIENCE

  • Bachelor’s degree or equivalent experience.
  • 7+ years of proven experience deploying and maintaining always-on data services.
  • 2+ years building and maintaining Spark notebook-based pipelines in Microsoft Fabric.
  • 2+ years of experience working with Microsoft Fabric
  • 2+ years of experience working with Microsoft Azure
  • 2+ years of experience with Delta Lake.
  • 1+ years of experience with SQL Server.
  • Paid Time Off, Paid Sick Leave, Paid Holidays
  • Competitive Medical, Dental, Vision, Short Term Disability, and Life Insurance
  • 401(k) plan with discretionary match available
  • Flexible Spending Account (FSA), Health Savings Account (HSA)
  • Voluntary benefits including Critical Illness, Group Accident, and Voluntary Life
  • Employee Referral Program
  • Exposure to a growing ecommerce company
  • Discounts on the GOVX website
  • Salary Range

    $120,000 - 140,000 Annually

    AAP / EEO Statement

    EOE. Veterans / Disabled.  Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.

    Position will require successful completion of a background check and drug testing prior to starting employment.

    About GOVX, Inc.

    Savings for Those Who Serve

    GOVX was founded in 2011 to offer exclusive benefits to those who serve our country. The GOVX membership is comprised of current and former members of the United States military, law enforcement, firefighting, medical services, and government personnel. We are dedicated to supporting these communities and to offering unique value to our members, while delivering an authentic platform for brands to reach our growing customer base. As the largest and fastest growing digital platform serving this deserving audience, we are committed to stretching the limits of ecommerce to deliver the best assortment for our members’ on-duty and off-duty needs.

    Create a job alert for this search

    Cloud Data Engineer • New York, New York, United States

    Related jobs
    Senior AWS Cloud Engineer / Lead (W2)New York City, NY

    Senior AWS Cloud Engineer / Lead (W2)New York City, NY

    Career Mentors, LLC • NEW YORK, NY, US
    Full-time
    Job Title : Senior AWS Cloud Engineer / Lead (W2).New York City, NY (Local Candidates Only).We are seeking an experienced and highly skilled. Senior AWS Cloud Engineer / Lead.This is a hands-on engin...Show more
    Last updated: 30+ days ago
    Data Engineer Manager

    Data Engineer Manager

    Wavestone • New York, NY, US
    Full-time
    Be part of a global consulting powerhouse, partnering with clients on their most critical strategic transformations.Energetic, solution-driven experts who focus as much on people as on performance ...Show more
    Last updated: 30+ days ago • Promoted
    BI Data Engineer

    BI Data Engineer

    United Nations FCU • Long Island City, NY, United States
    Full-time
    Data Engineer with 6+ years of experience in building data intensive applications, tackling challenging architectural and scalability problems with some exposure to gathering insights from data by ...Show more
    Last updated: 30+ days ago • Promoted
    Collibra Data Engineer- Onsite Role

    Collibra Data Engineer- Onsite Role

    SPAR Information Systems LLC • New York City, NY, United States
    Full-time
    Quick Apply
    Collibra Data Engineer- Onsite Role Location : NYC, NY Long Term Mandatory...Show more
    Last updated: 1 day ago
    Data Engineer

    Data Engineer

    MetroPlus Health Plan • New York, NY, United States
    Full-time +1
    Water Street, 7th Floor, New York, NY 10004 .New Yorkers by uniting communities through care.We believe that Health care is a right, not a privilege. If you have compassion and a collaborative sp...Show more
    Last updated: 30+ days ago • Promoted
    Hybrid Cloud Engineer

    Hybrid Cloud Engineer

    Openkyber • NY, United States
    Full-time
    Quick Apply
    ROLE LOCATION(S) : Hybrid : New York City, NY; Purchase, NY; Garden City, NY; Morristown, NJ; Conshohocken, PA; ROLE SUMMARY : We ...Show more
    Last updated: 6 days ago
    Staff Software Engineer - Data Cloud

    Staff Software Engineer - Data Cloud

    Rippling • New York, New York, United States, 10001
    Full-time
    Rippling is the first way for businesses to manage all of their HR & ITpayroll, benefits, computers, apps, and morein one unified workforce platform. By connecting every business system to one sourc...Show more
    Last updated: 30+ days ago
    Cost-Savvy Infra Engineer for Cloud & Datadog

    Cost-Savvy Infra Engineer for Cloud & Datadog

    Hex • New York, NY, United States
    Full-time
    A contemporary technology company in San Francisco seeks an Infra Engineer to dramatically reduce Datadog monitoring costs. Ideal candidates should have 3+ years in related engineering roles, expert...Show more
    Last updated: 7 days ago • Promoted
    Data Engineer

    Data Engineer

    Beauty by Imagination (BBI) • New York, NY, US
    Full-time
    Beauty by Imagination is a global haircare company dedicated to boosting self-confidence with imaginative solutions for every hair moment. We are a platform company of diverse, market-leading brands...Show more
    Last updated: 17 days ago • Promoted
    AWS Cloud Engineer City, NY

    AWS Cloud Engineer City, NY

    Career Mentors, LLC • New York, NY, US
    Full-time
    New York City, NY (Local Candidates Only).In-person interview is mandatory.The ideal candidate should be a strong communicator and possess deep expertise in cloud security, DevOps practices, and in...Show more
    Last updated: 30+ days ago
    Senior Data Engineer

    Senior Data Engineer

    IPG Mediabrands • New York, NY, United States
    Full-time
    KINESSO is the technology-driven performance marketing agency that sits at the very heart of IPG Mediabrands, providing actionable growth for both our agency partners and clients.We turn 'action' i...Show more
    Last updated: 30+ days ago • Promoted
    Senior Cloud Governance Engineer

    Senior Cloud Governance Engineer

    Openkyber • NY, United States
    Full-time
    Quick Apply
    Job Title : 151381- Data Warehouse Architect 5 Department : DTMB Work Location : 235 S Grand Ave, Lansing, MI 48933, US, (Hybrid) Projected Sta...Show more
    Last updated: 8 days ago
    Cloud Engineer / Cloud Architect- Associate Director

    Cloud Engineer / Cloud Architect- Associate Director

    KPMG • Montvale, NJ, United States
    Full-time
    Known for being a great place to work and build a career, KPMG provides audit, tax and advisory services for organizations in today's most important industries. Our growth is driven by delivering re...Show more
    Last updated: 23 days ago • Promoted
    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    OneStream Software • New York, NY, United States
    Full-time
    Senior AI Engineer - Cloud Engineering Focus.Hybrid, USA (Birmingham, MI, Chicago, IL, New York, NY, Fort Lauderdale, FL). Range applies to US candidates only) + Benefits / Variable Comp / Equity - Rang...Show more
    Last updated: 1 day ago • Promoted
    Cloud Data Architect

    Cloud Data Architect

    Tiger Analytics • Jersey City, NJ, US
    Full-time
    Quick Apply
    Tiger Analytics is an advanced analytics consulting firm.We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data.Our consultants b...Show more
    Last updated: 30+ days ago
    Cloud Platform Engineer

    Cloud Platform Engineer

    Hatch Global Search • New York, New York, US
    Full-time
    Quick Apply
    This is a hybrid role (3 days in the office / 2 days remote).About your team : The Enterprise Architecture organization is looking for a Cloud Platform Engineer to help shape the future of cloud inf...Show more
    Last updated: 30+ days ago
    Data Engineer

    Data Engineer

    Blue Orange Digital • New York, New York, US
    Full-time
    Quick Apply
    Blue Orange Digital is a data and AI consulting firm that helps companies turn complex data into real business outcomes.We partner with organizations across industries to design and deploy scalable...Show more
    Last updated: 30+ days ago
    Cloud Engineer

    Cloud Engineer

    Zone IT Solutions • New York, NY, US
    Full-time
    Quick Apply
    We are looking for a Cloud Engineer, where you will be a key player in shaping our cloud strategy and enhancing the services we provide to our clients. In this role, you will design, implement, and ...Show more
    Last updated: 30+ days ago