Hadoop Systems Administrator

JOB SUMMARY

As a Hadoop Systems Administrator, you will be responsible for daily operations and maintenance of IT systems centering around Continental's big data platform. You will be required to keep these IT systems maintained and optimized through daily, weekly, and monthly system tuning and periodic system maintenance. Day-to-day operations will require interfacing with other IT departments, developers, and project management. Manage testing and implementation of upgrades and changes to supported systems following change control processes.

DUTIES / RESPONSIBILITIES

• Day to Day maintenance activities including but not limited to log monitoring and cleanup, cluster monitoring, deployments, production support, and performance tuning.
• Install, configure, monitor, tune, and troubleshoot all components of the CDH environments, including but not limited to Cloudera Manager, Cloudera Management Services, HDFS, YARN, Zookeeper, Hive, Spark, Spark Streaming, Hue, Kudu, Impala, HBase, Key Management Server, Kafka, Flume, Solr, SSL, Sqoop, and Sentry.
• Build, tune, and administer search and index frameworks like Solr and Elastic Search.
• Build, tune, and administer HBase cluster in multiple environments.
• Design, document, develop, unit test, and deploy scripts for automating cluster patching, application upgrades, and application deployments using UNIX and Java in multiple environments.
• Automate daily administrative tasks to guarantee high availability of systems, applications, and data.
• Troubleshoot issues with applications, services, and clusters in multiple environments.
• Implement and administer active-active cluster configuration for disaster recovery.
• Manage cluster hardening activities with security implementation to enforce data governance for the data accessed by data scientists and analysts using Sentry and LDAP integration.
• Performance tune cluster services in multiple environments.
• Capacity planning for cluster upgrades.
• Setup monitoring and alerting for applications and services on the clusters using tools like Splunk and Grafana.
• Automate data and log backups and archival.
• Automate log analysis and setup alerting.
• Build, maintain, and administer clusters to support highly available and fast real-time streaming applications.
• Daily communication with business and technical teams to understand the needs and build, maintain, and support environments that help solve the use cases.
• Cluster patching activities.
• Cluster upgrades when required.
• Other duties assigned.

SKILLS

QUALIFICATIONS

J-18808-Ljbffr