Senior Big Data & DevOps Engineer
VDart IncPittsburgh, Pennsylvania, USA2 days ago
Job type
- Full-time
Job descriptionDesign manage and optimize HDFS directories tables and data partitioning strategies across large distributed clusters. Enforce data retention lifecycle policies and security compliance in high-volume Hadoop environments. Administer Hive and Impala platforms focusing on performance tuning capacity planning and troubleshooting. Develop scalable ETL pipelines and reusable data frameworks using PySpark Hive and Python. Implement and support CI / CD pipelines via Jenkins and uDeploy ensuring automated reliable multi-environment deployments. Manage platform upgrades patching and code promotion across DEV QA UAT and PROD stages. Perform advanced Linux administration including scripting system tuning disk management and disaster recovery. Drive incident and change management using ServiceNow including RCA investigation issue resolution and preventive actions. Mentor junior engineers foster technical leadership and document operational knowledge for internal sharing. Bachelors degree in Computer Science or related field. 8 years in Big Data engineering and DevOps. Advanced proficiency in HDFS Hive Impala PySpark Python and Linux. Proven CI / CD experience with Jenkins and uDeploy. Strong ETL development orchestration and optimization skills. Experience with ServiceNow for incident / change / problem management. Experience with cloud-based data platforms (AWS EMR). Knowledge of Docker Kubernetes and infrastructure automation (Ansible Terraform).
Role : Senior Big Data & DevOps Engineer.
Location : Pittsburgh PA (Remote).
Duration : Long Term Contract.
Position Summary :
- Seeking a Senior Big Data & DevOps Engineer with 8 years of expertise in HDFS Hive Impala PySpark Python and DevOps automation (uDeploy Jenkins). Responsible for end-to-end management of enterprise data platforms including HDFS table governance scalable ETL pipeline development CI / CD automation platform upgrades incident / change management and production support.
Key Responsibilities :
Required Qualifications :
Nice to Have :
Key Skills
Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala
Employment Type : Full Time
Experience : years
Vacancy : 1
Create a job alert for this search
Senior Data Engineer • Pittsburgh, Pennsylvania, USA
Related jobs
NET Developer / Azure Cloud Architect 37.Position is currently fully remote, but there's a chance it evolves into hybrid and / or full-time on-site...Show moreLast updated: 2 days ago
PLEASE READ : these jobs are testing jobs of Lever's testing environment - please do not apply for this job.Lever was founded ten years ago to tackle the most strategic challenge that companies face...Show moreLast updated: 30+ days ago
The Senior Manager, Business Development, will play a critical role in driving global sales and business development strategy for our cutting-edge Cell and Gene Therapies (CGT) Technologies divisio...Show moreLast updated: 30+ days ago
Zoho One Administrator / Developer.Potential for Contract Extension : Possible extension or contract-to-hire Initiatives / Projects.
Sales pathing and workflow automation.Supports merchant payment pro...Show moreLast updated: 1 day ago
Driving with Uber is an alternative to a part-time or full-time job and can get you earning cash on the road quickly.Driving with Uber allows you to earn quick cash while maintaining the flexibilit...Show moreLast updated: 22 hours ago
Govini transforms Defense Acquisition from an outdated manual process to a software-driven strategic advantage for the United States.
Our flagship product, Ark, supports Supply Chain, Science and Te...Show moreLast updated: 30+ days ago
The main function of a data engineer is to develop and implement data solutions for data platforms.Coordinate the collection, normalization and analysis of datasets across multiple data platforms.A...Show moreLast updated: 30+ days ago
Known for being a great place to work and build a career, KPMG provides audit, tax and advisory services for organizations in today's most important industries.
Our growth is driven by delivering re...Show moreLast updated: 20 days ago
Applying for this role online is quick and easy and you could even schedule your first day of work within 10 minutes!.Provide some basic information to start the application process.Watch a short ...Show moreLast updated: 11 hours ago
Taurus Industrial Group's Specialty Services division is seeking a dynamic Area / Territory Manager to lead operations and drive growth in the Pittsburgh Metro region.
This role requires a proven ...Show moreLast updated: 30+ days ago
At Edge Case, our mission is to enable the safe deployment of frontier technologies and sophisticated systems.We enable our customers to assure the safety of autonomous systems for real-world deplo...Show moreLast updated: 30+ days ago
Pittsburgh, PA (Hybrid mostly remote with occasional onsite collaboration).Contract To Hire, Seeking PGH based Candidates.
We're partnering with a growing SaaS company that's modernizing its platfor...Show moreLast updated: 12 days ago Pittsburgh, PA (Hybrid Remote with occasional on-site support).The Senior Azure Cloud Engineer plays a key role in designing, deploying, and managing modern cloud infrastructure within Microsoft en...Show moreLast updated: 30+ days ago PLEASE READ : these jobs are testing jobs of Lever's testing environment - please do not apply for this job.Lever was founded ten years ago to tackle the most strategic challenge that companies face...Show moreLast updated: 30+ days ago Driving with Uber is an alternative to a part-time or full-time job and can get you earning cash on the road quickly.Driving with Uber allows you to earn quick cash while maintaining the flexibilit...Show moreLast updated: 22 hours ago Driving with Uber is an alternative to a part-time or full-time job and can get you earning cash on the road quickly.Driving with Uber allows you to earn quick cash while maintaining the flexibilit...Show moreLast updated: 21 hours ago Earn at least $2790 driving with Uber when you complete your first 179 trips in 30 days.Driving is an easy way to boost your income while maintaining the flexibility your schedule requires (gig, pa...Show moreLast updated: 22 hours ago 
Siemens Energy NDE Project Engineer.Specialty Services product area is based at the Pittsburgh Service Center in Mount Pleasant, PA.
Specialty Services is responsible for performing non-destructive ...Show moreLast updated: 13 days ago
OA / ESO - TAS1
RICEFW Technologies IncHarrison, PA, United StatesFull-time
Quick Apply
Engineer
Lever Demo - IS OpportunitiesCalifornia, Pennsylvania, United States, 15419Full-time
- Promoted
Senior Manager, Business Development, Cell & Gene Therapies
Fresenius Kabi USA, LLCWarrendale, PA, United StatesFull-time
- Promoted
Technology Specialist - Zoho
System OnePittsburgh, PA, United StatesFull-time
- Promoted
- New!
Want to earn extra money? Drive with Uber
UberClaysville, PA, USFull-time +1
Senior Data Platform Engineer
GoviniPittsburgh, Pennsylvania, United States, 15201Full-time
Sr Data Engineer
Techstra SolutionsPittsburgh, PA, USFull-time
Quick Apply
- Promoted
Engineer, Data Visualization and Business Intelligence
KPMGPittsburgh, PA, United StatesFull-time
- Promoted
- New!
Warehouse Worker - $500 Sign-On Bonus
United Parcel ServiceConnellsville, PA, USFull-time +2
- Promoted
Area / Territory Manager
Taurus Industrial Group, LLCGreensburg, PA, USFull-time
Senior AI Engineer
Edge Case ResearchPittsburgh, PA, USFull-time
Quick Apply
DevOps Engineer
Prequel SolutionsPittsburgh, Pennsylvania, United StatesFull-time
Quick Apply
Sr Azure Cloud Engineer
Prequel SolutionsPittsburgh, Pennsylvania, United StatesFull-time
Senior Data Engineer
Lever Demo - IS OpportunitiesCalifornia, Pennsylvania, United States, 15419Full-time
- Promoted
- New!
Uber - Sign Up to Drive and Earn
UberWind Ridge, PA, USFull-time +1
- Promoted
- New!
Earn extra cash - Sign up and drive with Uber
UberClaysville, PA, USFull-time +1
- Promoted
- New!
Quick Earnings : Earn At Least $2790 For Your First 179 Trips, Guaranteed.
UberClaysville, PA, USFull-time +1
- Promoted
CT NDE Project Engineer
Siemens EnergyMount Pleasant, Pennsylvania, United States of AmericaFull-time