Cloudera Data Engineer - Remote

Job Description

Cloudera Data Engineer

Job Summary

We are seeking a   Cloudera Data Engineer   to support the migration of a Medicaid Data Warehouse Implementation in AWS environment. The resource will support the migration and continued operations of an existing   Cloudera / Hive / Scala-based data pipeline environment   from one AWS account to another.

This   position   is responsible for ensuring a seamless transition, validating data integrity and job performance, and maintaining reliable daily operations post-migration.

The role will work closely with the existing project team for the underlying AWS infrastructure (VPC, IAM, S3, EC2, networking). The resource will focus on   Cloudera cluster migration, data pipeline reconfiguration, and operational stability .

Key Responsibilities

• Replicate and configure existing Cloudera cluster (HDFS, YARN, Hive, Spark) in the new AWS account.
• Coordinate with project team to ensure proper infrastructure provisioning (EC2, security groups, IAM roles, and networking).
• Reconfigure cluster connectivity and job dependencies for the new environment.
• Migrate and validate metadata stores (Hive Metastore, job configs, dependencies).
• Validate job execution and data outputs for parity with existing environment.
• Deploy, test, and operate existing Hive, Spark (Scala) jobs post-migration.
• Maintain job schedules, dependencies, and runtime configurations.
• Monitor job performance, identify bottlenecks, and apply tuning or code-level optimizations.
• Troubleshoot failures and implement automated recovery or alerting where applicable.
• Monitor Cloudera Manager dashboards, cluster health, and resource utilization.
• Manage user roles and access within Cloudera environment.
• Implement periodic data cleanup, archiving, and housekeeping processes.
• Document configurations, migration steps, and operational runbooks.

Required Skills and Experience :

Preferred Qualifications :

Requirements

Qualifications Type Category Qualification Description Competency Required Skills Others Help desk support Help desk Novice (1-3 Years) No Skills Others Communication skills both verbal and written Novice (1-3 Years) No