Role Overview :
We are seeking a skilled Kafka Developer with deep expertise in Apache Kafka and Confluent Kafka platforms. The ideal candidate will have hands-on experience in designing, developing, and troubleshooting Kafka-based data pipelines, along with strong programming skills in Java or Scala. You will work closely with cross-functional teams to ensure reliable and scalable messaging systems across distributed environments.
Key Responsibilities :
- Troubleshoot Kafka messaging issues across producers, consumers, and brokers.
- Analyze Kafka logs to identify errors, warnings, and performance bottlenecks.
- Monitor Kafka clusters using tools to track metrics such as consumer lag, throughput, and partition health.
- Design and implement scalable Kafka-based data pipelines for real-time and batch processing.
- Collaborate with DevOps and data engineering teams to optimize Kafka configurations and deployments.
- Ensure high availability and fault tolerance through proper replication and partitioning strategies.
- Maintain and enhance Kafka topics, consumer groups, and broker configurations.
- Document Kafka architecture, data flow, and operational procedures.