Big data engineer Jobs in Phoenix az
Last updated: 2 days ago
- Promoted
Big Data Engineer
Zensar TechnologiesPhoenix, AZ, United StatesBig Data Engineer
Lorven TechnologiesPhoenix, Arizona, Florida, New Jersey, Pennsylvania, Texas, Virginia, United States Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
HarnhamPhoenix, AZ, United States- Promoted
- Promoted
- Promoted
- Promoted
Big Data Analyst / Developer (Python) - 82890
SwoonScottsdale, AZ, USBig Data Engineer
iSoftTek Solutions IncPhoenix, AZ, US- Promoted
GCP Big Data Engineer
Futran Tech Solutions Pvt. Ltd.Scottsdale, AZ, United StatesSenior Big Data Engineer
Highmark HealthAZ, Working at Home, ArizonaEngineer - Big Data - LUMI
American ExpressPhoenix, Arizona, United States- Promoted
- Promoted
Big Data Engineer with Java
Tekdoors IncPhoenix, AZ, United States of America- Promoted
- Promoted
Data Engineer
VirtualVocationsScottsdale, Arizona, United States- Promoted
Data Engineer AWSAirflowSnowflake
Tekvivid IncPhoenix, Arizona, USA- Promoted
GCP Data Engineer
Technogen International CompanyPhoenix, AZBig Data Engineer
Zensar TechnologiesPhoenix, AZ, United States28 days ago
Job type
- Full-time
Job descriptionExperience : experience working with Big Data technologies, particularly within the Hadoop Ecosystem (HDFS, Hive, Spark, HBase, Pig, Flume, etc.). Programming Skills : Strong programming experience in Java , Scala , or Python for data processing. Big Data Tools : Hands-on experience with Apache Spark , Hive , HBase , YARN , MapReduce , Kafka , and Flume . ETL Frameworks : Experience in building ETL pipelines and working with tools like Apache Nifi , Talend , or similar. Data Modeling & Warehousing : Knowledge of data warehousing , data modeling , and SQL (experience with NoSQL databases is a plus). Cloud Experience : Familiarity with cloud platforms such as AWS , Azure , or Google Cloud Platform and big data services like EMR , DataProc , etc. Data Integration : Experience integrating data from multiple sources and working with structured and unstructured data. Version Control & CI / CD : Familiarity with version control tools like Git and continuous integration / continuous deployment practices. Problem-Solving & Analytical Skills : Strong troubleshooting skills and ability to work on complex data problems in a fast-paced environment. Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes . Experience working with Apache Kafka for real-time data streaming. Understanding of machine learning concepts and integrating them into data pipelines (preferred but not required). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. Master’s degree is a plus.
Job Summary :
We are seeking a skilled and motivated Software Developer with a focus on Big Data Engineering to join our dynamic team. In this role, you will work with cutting-edge technologies in the Hadoop Ecosystem to develop, maintain, and optimize large-scale data processing systems. As part of our engineering team, you will be responsible for building robust data pipelines, ensuring data quality, and integrating big data solutions into our systems.
Key Responsibilities :
- Data Pipeline Development : Design, develop, and maintain scalable data pipelines using Hadoop, Spark, and related technologies to process and analyze large datasets.
- Big Data Systems : Work with Hadoop Ecosystem tools such as HDFS, Hive, HBase, and Pig to manage and optimize data storage, retrieval, and processing.
- ETL Design and Implementation : Build and optimize Extract, Transform, Load (ETL) workflows for high-volume data integration from various sources.
- Performance Optimization : Analyze and tune the performance of large-scale data processing workflows and jobs in a distributed computing environment.
- Collaboration : Collaborate with cross-functional teams, including Data Scientists, Analysts, and Product Managers, to gather requirements and deliver optimal data solutions.
- Data Quality & Security : Ensure data integrity, quality, and security by implementing robust validation checks, monitoring, and logging mechanisms.
- Innovation : Stay updated with the latest trends in Big Data technologies and continuously improve processes and systems with innovative solutions.
Required Skills & Qualifications :
Preferred Skills & Qualifications :
Education :