Senior Lead Data Engineer
BrillioSan Francisco, CA, United States14 hours ago
Job type
- Full-time
Job descriptionAzure Data Engineering Advanced : Senior Data Engineer Key Responsibilities : Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL Manage and optimize the infrastructure ensuring reliability and performance Design and implement data models and schemas for enterprise data warehouse and data marts Write and maintain scripts in Python for various applications Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases Work closely with analysts and product managers to understand data requirements and deliver high-quality data Provide technical guidance and mentorship to other team members fostering best practices in data engineering Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing Good knowledge in Python for data manipulation and scripting Mastery of advanced SQL and deep experience working with relational databases and datawarehouses Hands-on experience with workflow management tools like Apache Airflow Familiarity with Unix shell scripting for automation and system management
Senior Lead Data Engineer
Primary Skills
- AKS, Event Hub, Azure DevOps, Cosmos DB, Azure Functions
Specialization
Job requirements
Required Skills :
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Create a job alert for this search
Lead Data Engineer • San Francisco, CA, United States
Related jobs
This position can be 100% remote, but must be located in the United States.Dynavax is a commercial-stage biopharmaceutical company developing and commercializing novel vaccines to help protect the ...Show moreLast updated: 30+ days ago
It's what drives us to innovate.To continuously advance science and ensure everyone has access to the healthcare they need today and for generations to come.
Creating a world where we all have more ...Show moreLast updated: 30+ days ago
PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...Show moreLast updated: 30+ days ago
Alembic is pioneering a revolution in marketing, proving the true ROI of marketing activities.The Alembic Marketing Intelligence Platform applies sophisticated algorithms and AI models to finally s...Show moreLast updated: 30+ days ago
Trusted by 2,200+ financial institutions, Baselayer is the intelligent business identity platform that helps verify any business, automate KYB, and monitor real-time risk.Baselayer's B2B risk solut...Show moreLast updated: 30+ days ago
Together AI is looking for a Senior Data Engineer to help define, build, and operate the data infrastructure that handles millions of events every day to power Together's mission-critical systems.A...Show moreLast updated: 30+ days ago
At Apple, we focus deeply on our customers’ experience.Apple Ads brings this same approach to advertising, helping people find exactly what they’re looking for and helping advertisers grow their bu...Show moreLast updated: 30+ days ago
Finix is a full-stack acquirer processor, empowering businesses of all sizes with flexible, modern payment solutions.Processing billions of dollars annually, Finix enables SaaS, marketplace, and e-...Show moreLast updated: 30+ days ago
We exist to accelerate innovation.We do this by giving more people the opportunity to participate in the venture economy by building the financial infrastructure that makes it possible for more peo...Show moreLast updated: 30+ days ago
As an employee, you Turn Change Into Value® - for our clients, for our company, for your professional growth, for the consumers.
We hire the best and brightest, who are driven to create lasting valu...Show moreLast updated: 30+ days ago
Skydio is pioneering the future of autonomous flight, developing the worlds smartest drones for a wide range of applications.
In this role as a software engineer on our backend team, you will play a...Show moreLast updated: 14 hours ago
We're thrilled to embark on the search for a seasoned.Senior / Lead Data Solution Engineer.This pivotal role offers an exciting opportunity to shape the future of technology within our organization.A...Show moreLast updated: 30+ days ago
Robotics will have a massive positive impact on the world economy and global human productivity over the coming decade.At Foxglove, we're excited for this future, and we're building powerful open s...Show moreLast updated: 14 hours ago
PG Forsta is the leading experience measurement, data analytics, and insights provider for complex industries-a status we earned over decades of deep partnership with clients to help them understan...Show moreLast updated: 11 days ago
Hiring Manager : [Fardad Golshany] - Senior Data Engineer.We're looking for a data engineer with deep experience in analytics engineering and BI development to build the data infrastructure that pow...Show moreLast updated: 15 hours ago
REMOTE Senior Data Solutions Engineer / Lead Data Engineer Needed for Industry-Leader!.This Jobot Job is hosted by : Reed Kellick.
Are you a fit? Easy Apply now by clicking the "Apply Now" button and...Show moreLast updated: 9 days ago
Zum is a rapidly expanding Series E startup backed by industry leaders Sequoia Capital, SoftBank, Spark Capital, and GIC, with a bold mission to transform the stagnant school transportation industr...Show moreLast updated: 4 days ago Join to apply for the Senior / Lead Data Solution Engineer role at Meltwater.We're thrilled to embark on the search for a seasoned Senior / Lead Data Solution Engineer to join our vibrant team.This...Show moreLast updated: 15 hours ago
- Promoted
Senior Director, Data and AI Architecture Leader
Dynavax TechnologiesEmeryville, CA, United StatesFull-time
- Promoted
Senior Data Engineer
GenentechSouth San Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
Plum IncSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
Alembic LimitedSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
BASELAYERSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
Together AISan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
AppleSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
FinixSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
AngelListSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
xScion SolutionsSan Francisco, CA, United StatesFull-time
- Promoted
- New!
Senior Data Engineer
BoosterSan Mateo, CA, United StatesFull-time
- Promoted
Senior / Lead Data Solution Engineer
MeltwaterRedwood City, CA, United StatesFull-time
- Promoted
- New!
Senior Data Engineer
Foxglove Technologies, IncSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
PG ForstaEmeryville, CA, United StatesFull-time
- Promoted
- New!
Senior Data Engineer, Analytics
ScribeSan Francisco, CA, United StatesFull-time
- Promoted
Senior Data Engineer
JobotSan Jose, CA, USFull-time
- Promoted
Senior Engineer, Backend Platform
ZūmRedwood City, CA, USFull-time
- Promoted
- New!
Senior / Lead Data Solution Engineer
MeltwaterSan Francisco, CA, United StatesFull-time