DevOps Engineer

As an Analytics platform engineer supporting Azure & AWS Databricks from a DevOps perspective requires a hybrid skill set spanning primarily cloud infrastructure management and deployment, automation and CI/CD practices and it is helpful if they a background or exposure to ELT and data engineering and Database Administration Concepts etc.

The resource will be responsible developing the foundation Databricks for use within Bank as an Analytics Platform Engineering team member where they will develop automation via terraform modules or other tooling to deploy Databricks Workspaces, along with other cloud resources, we are developing a platform ontop of a PaaS offering where you also need to implement a CICD solution in Gitlab.
Core Responsibilities

• Implement CI/CD Pipelines: Design and maintain continuous integration and continuous deployment pipelines using tools like Gitlab, Azure DevOps or GitHub Actions for deploying data pipelines and infrastructure changes.

• Infrastructure as Code (IaC): Provision and manage Azure Analytics services (Databricks, etc.) through code using tools such as Terraform, ARM templates, or Bicep to ensure consistency and repeatability.

• Platform Operations: Deploy, manage, and optimize the performance and resource utilization of Azure data services, including monitoring and troubleshooting data pipeline failures and performance issues.

• Automation & Scripting: Automate routine operational tasks and application deployments using scripting languages like Python, PowerShell, or Bash.

• Security & Compliance: Implement security best practices, including identity and access management (IAM), data encryption, and compliance with data governance policies (, GDPR) within the platform.

• Collaboration: Work closely with data engineers, data scientists, and business analysts to translate data requirements into robust technical solutions and foster a DevOps culture within the organization
  
  
  Key Skills and Qualifications
  
  Cloud Platform Expertise: Deep knowledge of Microsoft Azure and/or AWS services, specifically, Azure and or AWS Databricks, Exposre to Azure Synapse Analytics (SQL pools, Spark pools, pipelines), Azure Data Factory (pipelines, triggers, data flows), would be beneficial as well. Azure Data Lake Storage (ADLS), and Azure Monitor
  DevOps Tools & Methodologies:
  
  CI/CD platforms: Expertise Gitlab or Azure Devlops (Azure Pipelines, Boards, Repos) or GitHub Actions.
  Version Control: Strong experience with Git and branching strategies.
  IaC tools: Proficiency in Terraform, Bicep, or ARM templates.
  Containerization: Experience with Docker and container orchestration (Azure Kubernetes Service - AKS) is highly beneficial.
  
  
  Programming & Scripting Languages: Proficiency in Python, SQL (SQL, query optimization), and scripting languages like PowerShell or Bash for automation and data manipulation tasks.
  Data Engineering Fundamentals: Basic understanding of Data engineering principals such as ETL/ELT processes, data modeling, data warehousing, and big data concepts.
  Monitoring & Logging: Experience implementing monitoring, logging, and alerting solutions using tools like Azure Monitor, Log Analytics, and Application Insights.
  Soft Skills: Excellent problem-solving, analytical, and communication skills to work effectively in fast-paced, agile environments.
  Preferred Certifications: Microsoft Certified: Azure Data Engineer Associate (DP-) or Microsoft Certified: Azure DevOps Engineer Expert (AZ-) are highly valued.
  
  

This Job can be out of any of the core Tech Hub Office's for USB. SF or Cupertino, California, Dallas Texas, Hopkins, MN, Atlanta GA, Charlotte NC.

------------------------DO NOT EDIT BELOW THIS LINE, PLEASE INSERT JOB DESCRIPTION ABOVE--------------------------------- The DevOps Engineer is responsible for developing, implementing and maintaining technical software applications and provides a combination of technical and business leadership while being the primary trusted and capable owner of one or more high priority, high visibility, complex initiatives. The DevOps Engineer will typically lead and coach a small number of team members (, an architect, systems analyst, etc.) and provide guidance to a larger community. ESSENTIAL FUNCTIONS: ? Designs and writes complex code in several languages relevant to our existing product stack, with a focus on automation ? Configures, tunes, maintains and installs applications systems and validates system functionality ? Monitors and fine tunes applications system to achieve optimum performance levels and works with hardware teams to resolve issues with hardware and software ? Lead, manage and grow a team of customer focused, highly dynamic and multi-disciplinary team of full stack software engineers ? Assists with application system problem resolution by working with application developers, vendors, and internal infrastructure teams member to troubleshoot ? Design and implement appropriate environments for DevOps tools set, and provide production support ? Influences other teams and broader Engineering groups in adopting Cloud technologies, processes, and best practices ? Automates the provisioning of environments for DevOps Tool chain ? Develops and enables continuous integration/continuous deployment (CI/CD) for DevOps Tool set ? Acts to improve the overall reliability of systems and to increase efficiency ? Works collaboratively with cross functional teams, using Agile / DevOps principles to bring products to life, achieve business objectives and serve customer needs ? Encourage innovation, implementation of cutting-edge technologies, outside-of-the-box thinking, teamwork, and self-organization

Shift: [' years of experience developing APIs and integrating with rd party APIs. Strong SQL skills (SQL Server or Oracle). Familiarity with scripting languages (Python, Bash, Powershell). Experience with version control systems (Git, GitHub, GitLab). Knowledge of CI/CD pipelines and DevOps best practices. Understanding of workflow automations is a plus. Excellent problem-solving, analytical, and communication skills.', 'Azure AD, OAuth, JWT, Microsoft Teams SDK, Building apps for teams, Azure hands with AD BC, BOT, SignalR, AKS, App Registrations, App Insights', 'Azure Databricks', 'CI/CD: Jenkins, Terraform, GitLab CI/CD', 'Terraform Module Development']

Start: ['Data Engineer', 'Strong understanding of ETL/ELT processes, data modeling, data warehousing, and big data concepts.']