Talent.com
Lead Cloudera Applications Engineer

Lead Cloudera Applications Engineer

KMM Technologies, Inc.Reston, VA, United States
7 hours ago
Job type
  • Full-time
  • Quick Apply
Job description

Position : Lead Cloudera Applications Engineer

Location : Reston, VA #Night Shift #Remote

Duration : 6+ months #C2H

IN-Person Interview

Rate : $80 / hr

Job Description :

  • Performs complex analysis, design, development, automated unit and integration testing, and debugging of computer software ranging from operating system architecture integration and software design to selection of computer systems, language, and / or equipment.
  • Designs, codes, tests, develops, and documents application programs for computer systems.
  • Works with end-users and project stakeholders to define system scope and objectives.
  • Manages modifications to and maintenance of operational programs and procedures.
  • Leads stakeholder and product owner meetings to define software requirements and creates highly complex systems. Acts as a mentor to junior engineers
  • Environment : Production Support Dept- need to work late nights and will be called in nights to work.
  • Monitoring the health of the Services on top of all clusters Big Data Support
  • Combination of Cloudera Techni with AWS
  • Supporting the Enterprise data Hub (Big Data / Hadoop databases) throughout the development and production lifecycle.
  • Bachelor's Degree (Information Technology or Computer Science)
  • 8 years Software design and development, software test and evaluation, software requirements management

Experience Summary :

  • Develop the scripts to Monitor the services on top of the Clusters and send out the Alerts if necessary.
  • Experience on the user Incidents who have issues with the Cloudera Services and assisting them accordingly.
  • Write the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
  • Monitoring the health of all the services running in the production cluster using the Cloudera Manager.
  • Performing / Accessing the databases, metastore tables and writing Hive, Impala queries using HUE.
  • Setting up the High Availability of the Services like Hue, Hive, HBase REST, SOLR and IMPALA on top of the all-new clusters that were build on the BDPaas Platform.
  • Responsible for monitoring the health of the Services on top of all clusters.
  • Working closely with different teams like Application development team, Security team, Platform Support to identify and implement the Configurational changes that are needed on top of the cluster for better performance of the services.
  • Skills Must Require :

    Areas of Expertise :

  • Experience in Supporting Cloudera Applications running in Cloud
  • Experience in Supporting Cloud Applications with Mongo DB
  • Experience in Cloud Disaster Recovery
  • Apache Kafka - strong Administration & troubleshooting skills
  • Kafka Streams API | Stream processing with KStreams & Ktables | Kafka integration with MQ

    Kafka broker management | Topic / offset management | Apache Nifi - Administration

    Flow management | registry server management | controller service management

    Nifi to kafka / Hbase / solr integration | Flume - integration with Kafka, Nifi & IBM MQ

    Hbase - administration | database management | troubleshooting | Solr - administration

    managing Logging level | managing shards & high availability | Collection management

    Rectify resource intensive & long running solr queries

    Additional Skills includes :

  • Ensure Cloudera installation and configuration is at optimal specifications (CDP, CDSW, Hive, Spark, NiFi).
  • Perform critical data migrations from CDH to CDP.
  • Design and implement big data pipelines and automated data flows using Python / R and NiFi.
  • Assist and provide expertise as it pertains to automating the entire project lifecycle.
  • Perform incremental updates and upgrades to the Cloudera environment.
  • Assist with new use cases (i.e., analytics / ML, data science, data ingest and processing), Infrastructure (including new cluster deployments, cluster migration, expansion, major upgrades, COOP / DR, and security).
  • Assist in testing, governance, data quality, training, and documentation efforts.
  • Move data and use YARN to allocate resources and schedule jobs.
  • Manage job workflows with Oozie and Hue.
  • Implement comprehensive security policies across the Hadoop cluster using Ranger.
  • Configure and manage Cloudera Data Science Workbench using Cloudera Manager.
  • Troubleshoot potential issues with Kerberos, TLS / SSL, Models, and Experiments, as well as other workload issues that data scientists might encounter once the application is running.
  • Supporting the Big Data / Hadoop databases throughout the development and production lifecycle.
  • Troubleshooting and resolving database integrity issues, performance issues, blocking and deadlocking issues, replication issues, log shipping issues, connectivity issues, security issues, performance tuning, query optimization, using monitoring and troubleshooting tools.
  • Create, test, and implement scripting for automation support.
  • Experience in working with Kafka ecosystem (Kafka Brokers, Connect, Zookeeper) in production is ideal
  • Implement and support streaming technologies such as Kafka, Spark & Kudu
  • ESSENTIAL FUNCTIONS :

    25% Lead a team of software engineers in the design, development, and implementation of multiple applications. Performs and leads the detail technical design, coding, code review and testing of new applications and enhancements of existing applications. Serves as a subject matter expert for specified systems and their subsequent applications. Develop and / or analyze interface design documentation. Comply with hardware and software systems standards and procedures. Deliver large systems for tens of thousands of users. Ensures compliance to the dev / sec / ops practices and policies. Understanding of architectural practices and design.

    25% Identifies gaps in requirements and solves problems for the supporting staff. Identifies sequencing of services and technologies that are integrated in later phases of the project. Leads the evaluation of risk and opportunity impacts to existing or proposed projects / programs and implements risk mitigation actions. Develops ways to improve performance and increase system availability. Provides recommendations to infuse new technology. Compile, assess, and report data results from software testing and analysis.

    25% Supports the Project Manager as the technical team lead on the project. Estimates the technical costs of projects for clients and project managers. Confers with clients to identify requirements and determine desired outcomes to formulate the design of the system or offer alternatives. Demonstrated experience with front-end and back-end programing and integration frameworks and tools. Assist in the estimation of development of application features.

    25% Leads software design and development teams. Identifies, analyzes, defines, and coordinates user, customer, and stakeholder needs and translates them into technical requirements. Manages technical resources (hardware, software, licenses, and services). Plans and leads transformation initiatives to new software architectures. Extend open-source libraries and systems as necessary to meet the needs of the team. Play a crucial role in product scoping and roadmap / architecture discussions. Continually evaluate emerging technologies to identify opportunities, trends, and best practices. Ensure code quality, deployment standards and practices. Mentoring junior developers.

    Thanks & Regards,

    LAXMAN

    Team Lead - Talent Acquisition

    KMM Technologies, Inc.

    CMMI Level 2 | ISO 9001 | ISO 20000 | ISO 27000 Certified

    WOSB, SBA 8(A), MDOT MBE & NMSDC MBE

    Contract Vehicles : 8(a) STARS III & Schedule 70

    Tel : (240) 800-0039 |Email : lax@kmmtechnologies.com

    Create a job alert for this search

    Application Engineer • Reston, VA, United States

    Related jobs
    • Promoted
    DevOps Cloud Engineer

    DevOps Cloud Engineer

    ALTA IT ServicesAlexandria, VA, US
    Full-time
    Job Title : DevOps Cloud Engineer Location : Alexandria, VA DevOps Cloud Engineer Top Secret clearance $180k Alexandria, VA or Gaithersburg, MD We are looking for an experienced DevOps Cloud Network ...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    Leidos IncChantilly, VA, United States
    Full-time
    Leidos Intelligence Group is seeking a highly technical Cloud Engineer to support an enterprise IT program.This work is being done to replace an end of life solution with a new solution using new a...Show moreLast updated: 30+ days ago
    • Promoted
    Hybrid Cloud Engineer Senior

    Hybrid Cloud Engineer Senior

    Leidos IncReston, VA, United States
    Full-time
    The Leidos Digital Modernization Sector is seeking a highly skilled Azure Cloud Engineer with deep expertise in networking and infrastructure automation. This is a 100% remote hands-on engineering r...Show moreLast updated: 25 days ago
    • Promoted
    Mid-Level Cloud Platform Engineer

    Mid-Level Cloud Platform Engineer

    Red Gate GroupReston, VA, US
    Full-time
    The Red Gate Group, founded in 2006, is a Service-Disabled Veteran-Owned Company based in Chantilly, VA.As an SDVOSB, the Red Gate Group has established a reputation for delivering quality multi-ti...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    GSSR IncRockville, MD, US
    Full-time
    Applicantsmust be local to DC area or willing to relocate.Weare seeking a skilled Cloud Engineer to design, implement, and maintaincloud-based solutions using AWS tools, Docker, Fargate, containeri...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering)

    Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering)

    Capital OneWashington, DC, US
    Full-time +1
    Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering).Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-pa...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    The Aerospace CorporationChantilly, VA, United States
    Full-time
    The Aerospace Corporation is the trusted partner to the nation's space programs, solving the hardest problems and providing unmatched technical expertise. As the operator of a federally funded resea...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Cloud Data Platform Engineer (Azure / AWS)

    Cloud Data Platform Engineer (Azure / AWS)

    LeidosAlexandria, VA, US
    Full-time
    B;At Leidos, we're not just delivering solutions – we're pioneering the future of defense and intelligence technology.Our diverse teams of innovators unite around a singular purpose : ...Show moreLast updated: 21 hours ago
    • Promoted
    Lead Data Engineer (Cloud Operations Resilience Engineering)

    Lead Data Engineer (Cloud Operations Resilience Engineering)

    Capital OneMcLean, VA, US
    Full-time +1
    Lead Data Engineer (Cloud Operations Resilience Engineering).Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborati...Show moreLast updated: 28 days ago
    • Promoted
    Linux Cloud Engineer

    Linux Cloud Engineer

    PeratonChantilly, VA, United States
    Full-time
    Chantilly, VA office in support of our Department of Defense (DoD) Community customer as part of a highly talented, highly motivated, and high-performing team. As part of the systems engineering tea...Show moreLast updated: 3 days ago
    • Promoted
    Datacenter and Cloud Engineer

    Datacenter and Cloud Engineer

    Cooley LLPWashington, DC, United States
    Full-time
    Cooley is seeking a Datacenter and Cloud Engineer to join the.Technology Infrastructure & Development Operations.The Data Center and Cloud Engineer ("DCE") is responsible for the firm's Datacenters...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    BOEINGArlington, VA, US
    Permanent +1
    At Boeing, we innovate and collaborate to make the world a better place.We’re committed to fostering an environment for every teammate that’s welcoming, respectful and inclusive, with g...Show moreLast updated: 7 days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    LeidosTakoma Park, MD, US
    Full-time
    Leidos has an exciting opening for you, as a Cloud Engineer, to play a critical role with design, engineering, development, deployment, and use of software for the DOMEX Technology Platform (D...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Cloud DataOps Engineer

    Senior Cloud DataOps Engineer

    SurescriptsArlington, VA, United States
    Full-time
    We deliver insights at critical points of care for better decisions - from streamlining prior authorizations to delivering comprehensive medication histories to facilitating messages between provid...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Application Engineer

    Cloud Application Engineer

    NextGen Solutions CorpWashington, DC, US
    Full-time
    Please note that this position is with our direct client.Please note this position is 100% ONSITE.Manages, organizes, and administers cloud-based systems analysis while designing and developing bus...Show moreLast updated: 30+ days ago
    Lead Cloud Engineer (MD)

    Lead Cloud Engineer (MD)

    Cooperidge Consulting FirmLaurel, MD, US
    Full-time
    Quick Apply
    Cooperidge Consulting Firm is seeking a skilled Lead Cloud Engineer to oversee the design, implementation, and management of cloud-based solutions for one of our DoD clients.In this role, you will ...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Senior Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering)

    Senior Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Lead Software Engineer, Full Stack (Cloud Operations Resilience Engineering) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a ...Show moreLast updated: 6 hours ago
    • Promoted
    Manager, Cloud Applications

    Manager, Cloud Applications

    HITT ContractingFalls Church, VA, United States
    Full-time
    Every day, our team members do amazing things in pursuit of our shared purpose to build trust with our clients, partners, subcontractors, and teammates. No matter your background, education, or care...Show moreLast updated: 19 days ago