Big Data Lead

DatenSnowflake, AZ, United States

4 days ago

Job type

Full-time

Job description

Overview

We are seeking an experienced Big Data Lead to design, implement, and optimize large-scale data engineering solutions on modern cloud and distributed computing platforms. The role requires strong technical expertise in Big Data frameworks, advanced SQL, data integration, and cloud-native services. The ideal candidate will have proven experience in leading data projects, building scalable data pipelines, and delivering actionable insights through analytics and visualization platforms.

Education Requirements

Bachelors degree in computer science or a related technical field.
Minimum 5 years of relevant experience in Big Data Engineering / Data Analysis.

Experience Requirements

Strong proficiency in Python, SQL, and Apache Spark (Core Spark, Spark Streaming, DataFrame API, DataSet API, RDD, Spark SQL).

Hands-on experience with AWS services including EMR, Glue (serverless architecture), S3, Athena, IAM, Lambda, and CloudWatch.

Expertise in Hadoop and distributed computing frameworks.

Advanced SQL development with Hive / Impala including performance tuning.

Experience with ElasticSearch (OpenSearch) and Kibana dashboards .

Familiarity with resource management frameworks such as Yarn or Mesos.

Knowledge of external job schedulers like Autosys, AWS Data Pipeline, or Airflow.

Experience in HBase and other Key / Value data stores.

Role & Responsibilities

Design, develop, and maintain scalable data pipelines across Big Data platforms.

Perform physical data modeling and table design for high-volume data systems.

Build and manage data catalogs and dictionaries to maintain data integrity.

Deliver solutions using AWS services (EMR, Glue, Lambda, S3, Athena, Redshift, Snowflake).

Optimize big data workloads leveraging Hadoop, Spark, and cloud-native frameworks.

Data Integration & ETL

Clean, transform, and blend data from multiple sources using ETL tools.

Implement automation for data ingestion, transformation, and monitoring processes.

Deliver self-service analytics, KPIs, and dashboards for business teams.

Develop Tableau dashboards, including calculated fields and interactive reports.

Support advanced analytics by integrating ElasticSearch / OpenSearch with Kibana.

Work closely with cross-functional teams to define business requirements and translate them into technical data solutions.

Lead and mentor a team of engineers to deliver high-quality data-driven outcomes.

Ensure effective communication between business stakeholders, data engineering, and analytics teams.

Performance & Optimization

Conduct SQL and query optimization for high-performance analytics.

Monitor data pipelines, troubleshoot performance bottlenecks, and implement scalable solutions.

Perform root cause analysis and ensure reliability of data platforms.

#J-18808-Ljbffr

Create a job alert for this search

Data Lead • Snowflake, AZ, United States

Related jobs

Promoted

Share Your Thoughts, Pocket Some Cash : Join Opinion Outpost's Survey Opportunities!

DynataShow Low, AZ, US

Full-time

Ready to turn your opinions into cash? Explore the world of remote paid online survey gigs with Opinion Outpost!.It's as simple as sharing your thoughts from the comfort of your own home and watchi...Show moreLast updated: 4 days ago

Promoted

Cloud Data Operations Engineer (Business Financing)

Coherent SolutionsSnowflake, AZ, United States

Full-time

SolutionsAt Coherent Solutions, we offer engineering solutions that protect your time, your bottom line, and most importantly, your business. Our ApproachWe work collaboratively and transparently, p...Show moreLast updated: 1 day ago

Promoted

Freelance Data Collector

Prime InsightsShow Low, AZ, US

Full-time

Join thousands of members already earning with top-paying surveys and offers.Get started today and enjoy competitive rewards, fast payouts with no waiting periods, and the flexibility to participat...Show moreLast updated: 4 days ago

Promoted

Equity Bank Need Associate BI Developer and Engineer

Opportunities For Young KenyansSnowflake, AZ, United States

Full-time

Equity Bank Need Associate BI Developer and Engineer.The Associate BI and Data Developer plays a foundational role in the design, development, and optimization of Business Intelligence and data pro...Show moreLast updated: 1 day ago

Promoted

Online Survey Participant

Prime InsightsShow Low, AZ, US

Full-time

Promoted

Earn Extra Income by Taking Surveys Online

Prime InsightsShow Low, AZ, US

Full-time

Promoted

Side Hustle Project Lead

Finance BuzzTaylor, Arizona, US

Full-time +1

We’re offering a role for someone who wants to lead their own side-income project in their spare time.You’ll explore various proven side hustles, select the ones that fit your lifestyle, and run th...Show moreLast updated: 26 days ago

Promoted

Sr. ETL Engineer

ACLdigitalSnowflake, AZ, United States

Full-time

In-depth, hands-on experience with data warehouse ETL tools and technologies such as Abinitio, Snowflake / Oracle, Control-M / Airflow that will be leveraged to support our ETL platform built on AWS.Th...Show moreLast updated: 2 days ago

Shift Leader

Panda Restaurant GroupShow Low, AZ, US

Full-time

Our Panda Shift Leader associates are important leaders of our team and are responsible for bringing Panda’s mission alive in our restaurants by supporting management in creating food with passion,...Show moreLast updated: 30+ days ago

Promoted

Sr. Cloud Data Warehouse Architect

The Phia GroupSnowflake, AZ, United States

Full-time

The Phia Group is comprised of The Phia Group, LLC and Phia Group Consulting, LLC (PGC), and is a service-oriented organization assisting employee health plans nationwide.We provide our clients wit...Show moreLast updated: 2 days ago

Promoted

Senior Snowflake Developer

QtsolvSnowflake, AZ, United States

Full-time

Senior Snowflake Developer • Snowflake • Matillion • Airflow • Snowpark • DBT • PythonExperience Level : 5+ years of overallexperience. Candidate's recentproject must be on Snowflake.Python, Matillion & Sn...Show moreLast updated: 6 days ago

Promoted

Senior SDET, Data QA [Remote-US]

Quanata, LLCSnowflake, AZ, United States

Remote

Full-time

Were looking for an accomplished and well-rounded Senior SDET with strong experience in testing complex.This role requires a hands-on engineer who can build robust automated test frameworks, work c...Show moreLast updated: 1 day ago

Promoted

Fully Remote Survey Participant

Prime InsightsShow Low, AZ, US

Remote

Full-time

Promoted

AI Senior Developer Senior Python Software Engineer

HeadHRSnowflake, AZ, United States

Full-time

Data Science # Python # LLM # Azure # AI # Communication skills # Problem solving.Data Science # Python # LLM # Azure # AI # Communication skills # Problem solving. AI Developer (m / f / d) for Customer...Show moreLast updated: 1 day ago

Promoted

County Meter Reader

Meter ReaderJoseph City, AZ

Full-time

Responsibilities The primary responsibility of this position is to read meters and record consumption of the water used, cleaning of meter boxes, and removal of vegetation impeding access to meters...Show moreLast updated: 4 days ago

Promoted

Sr. Data Quality Engineer

The Kynetec GroupSnowflake, AZ, United States

Full-time

Develop, implement, and manage data quality processes to ensure accuracy, completeness, and reliability of data from machine learning models, data science processes, and external sources.Create and...Show moreLast updated: 6 days ago

Promoted

Senior Backend Engineer Data & Experimentation Systems

Ameware GroupSnowflake, AZ, United States

Full-time +1

When you choose to work with Ameware Group, you work with industry experts who are willing to share their knowledge and experience. You will be able to broaden your horizons by working in an open an...Show moreLast updated: 6 days ago

Promoted

Take Surveys Get Paid!

Prime InsightsShow Low, AZ, US

Full-time