Big Data Developer

We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.

Key Responsibilities

• Develop, maintain, and optimize big data applications and data processing pipelines.
• Work with Kafka and Apache Spark for real-time and batch data processing.
• Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
• Write and optimize complex SQL queries using Hive and Impala.
• Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
• Use Splunk for log analysis, monitoring, and troubleshooting.
• Manage and monitor batch workflows and job scheduling using AutoSys.
• Work in Linux-based environments for development, deployment, and operational support.
• Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
• Ensure data quality, reliability, scalability, and performance of big data applications.

Required Technical Skills

• Strong hands-on experience with Kafka, Spark, Scala, and SQL.
• Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
• Proficiency in data analysis and data wrangling for large datasets.
• Experience in performance analysis, tuning, troubleshooting, and issue resolution.
• Hands-on experience with Splunk and AutoSys.
• Strong knowledge of Linux-based infrastructures.
• Experience in distributed data processing and big data technologies.