JOB DESCRIPTION
Key Responsibilities :
- Install, configure, and maintain Apache Kafka clusters.
- Manage Kafka topics, partitions, consumer groups, and brokers.
- Implement and maintain Kafka security using SSL, SASL, and ACLs.
- Monitor Kafka performance and availability using tools like Prometheus, Grafana, or Confluent Control Center.
- Perform routine cluster maintenance, including upgrades, patches, and performance tuning.
- Troubleshoot Kafka-related issues and perform root cause analysis.
- Automate Kafka operations and support CI / CD pipelines.
- Collaborate with developers, DevOps, and data engineering teams to support real-time data pipelines and event-driven architectures.
- Document Kafka infrastructure and standard operating procedures.
Required Skills and Qualifications :
2-5+ years of experience managing Apache Kafka in production environments.Strong knowledge of Kafka internals (brokers, zookeepers, producers, consumers, etc.).Experience with Kafka administration tools (e.g., Kafka Manager, Confluent Platform).Familiarity with distributed systems and high availability setups.Proficiency in scripting languages like Bash, Python, or Shell.Experience with Linux / Unix systems and networking.Understanding of monitoring and alerting tools (e.g., Prometheus, Grafana, Splunk).Knowledge of Kafka security features (encryption, authentication, authorization).Hands-on experience with DevOps tools like Docker, Kubernetes, Ansible, Jenkins is a plus.Preferred Qualifications :
Experience with Confluent Kafka or other Kafka cloud services (AWS MSK, Azure Event Hubs, etc.).Familiarity with data pipeline tools such as Kafka Connect, Kafka Streams, or Apache Flink.Certifications in Kafka (e.g., Confluent Certified Administrator) are a plus.Bachelor's degree in Computer Science, Information Technology, or a related field.