Talent.com
Lead Data Engineer
Lead Data EngineerUniversity of Texas at Austin • Austin, TX, United States
Lead Data Engineer

Lead Data Engineer

University of Texas at Austin • Austin, TX, United States
15 hours ago
Job type
  • Full-time
  • Temporary
Job description

Job Posting Title : Lead Data Engineer

Hiring Department :

Enterprise Technology - Data to Insights (D2I)

Position Open To : All Applicants

Weekly Scheduled Hours :

40

FLSA Status : Exempt

Earliest Start Date :

Immediately

Position Duration :

Expected to Continue Until Dec 19, 2026

Location : Texas

Job Details : General Notes

This is a fixed term position that is expected to continue for a 1-year limited term from start date with a possibility for extension.

Flexible work arrangements are available for this position, including the ability to work 100% remotely. Remote work for individuals who reside outside Texas but within the United States and its territories will be considered and requires Central Office approval.

This position provides life / work balance with typically a 40-hour work week and travel limited to training (e.g., conferences / courses).

Enterprise Technology is dedicated to supporting the mission of the University of Texas at Austin of unlocking potential and preparing future leaders of the state.

Your skills will make a difference.

You'll be working for a university that is internationally recognized for research and the work you do will make a difference in the lives of our students, faculty and staff. If you're the type of person that wants to know your work has meaning and impact, you'll like working for our campus.

The University of Texas at Austin and Enterprise Technology provide an outstanding benefits package to our staff. Those benefits include :

Competitive health benefits (Employee premiums covered at 100%; family premiums at 50%)

Vision, dental, life, and disability insurance options

Paid vacation, sick leave, and holidays

Teachers Retirement System of Texas (a defined benefit retirement plan)

Additional voluntary retirement programs : tax sheltered annuity 403(b) and a deferred compensation program 457(b)

Flexible spending account options for medical and childcare expenses

Training and conference opportunities

Tuition assistance

Athletic ticket discounts

Access to UT Austin's libraries and museums

Free rides on all UT Shuttle and Capital Metro buses with staff ID card

For more details, please see : https : / / hr.utexas.edu / prospective / benefits and https : / / hr.utexas.edu / current / services / my-total-rewards

Must be authorized to work in the United States on a full-time basis for any employer without sponsorship.

This position requires you to maintain internet service and a mobile phone with voice and data plans to be used when required for work.

Purpose

The Lead Data Engineer for the UT Data Hub improves university outcomes and advances the UT mission to transform lives for the benefit of society by increasing the useability and value of institutional data. You will lead senior data engineers and data engineers to create complex data pipelines within UT's cloud data ecosystem in support of academic and administrative needs. In collaboration with our team of data professionals, you will help build and run a modern data hub to enable advanced data-driven decision making for UT. You will leverage your creativity to solve complex technical problems and build effective relationships through open communication within the team and outside partners.

Responsibilities

Technical Leadership :

  • Design, architect, and deliver production-grade, scalable data pipelines and AI-ready data platforms using Databricks,AWScloud-native services and modern data engineering frameworks.
  • Lead end-to-end implementation oflakehousedata pipelines, ensuring performance, reliability, and cost efficiency.
  • Championindustrybest practices for data engineering.
  • Conduct andparticipatein peer code reviews tomaintaincode quality and consistency across the team.
  • Proactivelyidentifyand resolve bottlenecks in data ingestion, transformation, and orchestration processes using Databricks Delta Live Tables, Spark optimizationtechniques, and workflow automation.
  • Implement systems for data quality, observability, governance, and compliance using tools such as Unity Catalog, Delta Lake, and data validation frameworks.
  • Lead technical knowledge-sharing sessions on topics such as AI / ML integration, datalakehousearchitecture, and emerging data technologies.

Project Management :

  • Define project milestones, timelines, and deliverables for data and AI initiatives, ensuringtimelyand high-quality outcomes.
  • Collaborate with both internal and external stakeholderssuch as data architects, system architects, business users, Agile teammembers, and other D2I internal groups.
  • Manage project priorities, sprint planning, and team workloads while balancing innovation with delivery.
  • Communicate risks, dependencies, and resource constraints effectively, and develop mitigation plans for on-time project delivery.
  • Team Management and Leadership :

  • Supervise and mentor a team of Data Engineers (2-5 individuals) working on cloud, Databricks, and AI pipeline initiatives.
  • Foster a culture of continuous learning, experimentation, and technical excellence, encouraging engineers to explore AI and automation use cases.
  • Participate in recruiting, onboarding, and developing data engineering talent with strong Databricks and AI skillsets.
  • Conduct performance reviews, set development goals, and create individualized growth plans for team members.
  • Encourage collaboration across Data, AI / ML, Analytics, and Infrastructure teams to drive cross-functional success.
  • Communication :

  • Provide regular updates on project progress, technical challenges, andprojectmilestones to both technical and business stakeholders.
  • Translate complex technical concepts related to Databricks, AI, and data architecture into clear narratives for non-technical audiences.
  • Foster a transparent communication culture and provide actionable feedback to promote a growth mindset.
  • Ensure all data engineering processes, architectures, and standards are well-documented for reuse, governance, andknowledgecontinuity.
  • Innovation and Other Duties :

  • Stay current with advancements in AI, data engineering, and Databricks ecosystem, evaluating new tools and frameworks for potential adoption.
  • Pilot and promote innovative solutions such as AI-assisted data quality checks, data observability automation, and intelligent pipeline optimization.
  • Perform other duties as assigned, contributing to the organization's data-driven and AI-enabled transformation.
  • Required Qualifications

  • Bachelor's orMaster's degree in Computer Science, Information Systems, Engineering, or equivalent professional experience.
  • 5+ years of experience designing, implementing, andoptimizingcomplex, production-grade datapipelinesor enterprise-scale data platforms.
  • 5 years of experience in cloud-based data engineering using Databricks and Amazon Web Services (AWS) (e.g., Glue, S3, Lambda, Redshift).
  • 3+ years of experience managing or leading teams of data and / or software engineers, including mentorship, performance management, and project delivery.
  • Expertisein Python,PySpark, and SQL, withstrongunderstanding of data modeling, stored procedures, and scalable data transformations.
  • Proven experience architecting and implementing ETL / ELT solutions across relational, non-relational, andlakehouseenvironments (e.g., Delta Lake, Parquet, or Iceberg).
  • Experience designing and managing CI / CD pipelines andinfrastructureascode(IaC) using tools such as Databricks Repos, CDK, Terraform, or GitHub Actions.
  • Demonstrated knowledge of test-driven development (TDD) and data quality frameworks, ensuring reliability and reproducibility across data workflows.
  • Deep understanding of data governance, security, and compliance standards in cloud environments.
  • Excellent analytical, problem-solving, and debugging skills across distributed data systems.
  • Proven ability to communicate complex technical concepts clearly to both technical and non-technical audiences.
  • Experience supervising, mentoring, and guiding junior team members on technical and professional development.
  • Equivalent combination of relevant education and experience may be substituted as appropriate.

    Preferred Qualifications

  • 8+ years of experience in Data Engineering or related fields, including 5+ years of hands-on experience building andoptimizingdata pipelines on Databricks or similar large-scale data platforms.
  • Proven experience implementinglakehousearchitectures leveraging Databricks Delta Lake, Delta Live Tables, and Unity Catalog for governance and scalability.
  • Experience designing AI-ready data platforms and integrating machine learning pipelines using tools such asMLflowor model registry frameworks.
  • 3+ years of experience managing or leading cross-functional technical teams, fostering collaboration between Data Engineering, Analytics, and AI / ML teams.
  • 5+ years of experience with Agile software development methodologies and project tracking systems such as JIRA.
  • Expertisein distributed data processing and streaming frameworks, such as Apache Spark, Kafka, Flink, or Airflow, for orchestration and automation.
  • Strong familiarity with data observability, cost optimization, and performance tuning in Databricks and cloud-nativearchitecture.
  • Professional certifications such as Databricks Certified Data Engineer Professional or AWS Solutions Architector AWSData Analytics Specialty are highly desirable.
  • Demonstrated ability to introducenew technologiesand best practices to modernize existing data environments and promote AI / analytics maturity across the organization.
  • Passion for continuous learning and staying current with emerging technologies in data engineering, AI integration, and Databricks ecosystem advancements.
  • Salary Range

    $125,000 - $143,712

    Working Conditions

  • May work around standard office conditions
  • Repetitive use of a keyboard at a workstation
  • Use of manual dexterity (ex : using a mouse)
  • Work Shift

  • Monday - Friday 8am-5pm; Occasional nights or weekends may be required
  • Required Materials

  • Resume / CV
  • 3 work references with their contact information; at least one reference should be from a supervisor
  • Letter of interest
  • Importantfor applicants who are NOT current university employees or contingent workers : You will be prompted to submit your resume the first time you apply, then you will be provided an option to upload a new Resume for subsequent applications. Any additional Required Materials (letter of interest, references, etc.) will be uploaded in the Application Questions section; you will be able to multi-select additional files. Before submitting your online job application, ensure thatALLRequired Materials have been uploaded. Once your job application has been submitted, you cannot make changes.

    Important for Current university employees and contingent workers : As a current university employee or contingent worker, you MUST apply within Workday by searching for Find UT Jobs. If you are a current University employee, log-in to Workday, navigate to your Worker Profile, click the Career link in the left hand navigation menu and then update the sections in your Professional Profile before you apply. This information will be pulled in to your application. The application is one page and you will be prompted to upload your resume. In addition, you must respond to the application questionspresented to upload any additional Required Materials (letter of interest, references, etc.) that were noted above.

    Employment Eligibility :

    Regular staff who have been employed in their current position for the last six continuous months are eligible for openings being recruited for through University-Wide or Open Recruiting, to include both promotional opportunities and lateral transfers. Staff who are promotion / transfer eligible may apply for positions without supervisor approval.

    Retirement Plan Eligibility :

    The retirement plan for this position is Teacher Retirement System of Texas (TRS), subject to the position being at least 20 hours per week and at least 135 days in length.

    Background Checks :

    A criminal history background check will be required for finalist(s) under consideration for this position.

    Equal Opportunity Employer :

    The University of Texas at Austin, as an equal opportunity / affirmative action employer ,complies with all applicable federal and state laws regarding nondiscrimination and affirmative action. The University is committed to a policy of equal opportunity for all persons and does not discriminate on the basis of race, color, national origin, age, marital status, sex, sexual orientation, gender identity, gender expression, disability, religion, or veteran status in employment, educational programs and activities, and admissions.

    Pay Transparency :

    The University of Texas at Austin will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor's legal duty to furnish information.

    Employment Eligibility Verification :

    If hired, you will be required to complete the federal Employment Eligibility Verification I-9 form. You will be required to present acceptable and original documents to prove your identity and authorization to work in the United States. Documents need to be presented no later than the third day of employment. Failure to do so will result in loss of employment at the university.

    E-Verify :

    The University of Texas at Austin use E-Verify to check the work authorization of all new hires effective May 2015. The university's company ID number for purposes of E-Verify is 854197. For more information about E-Verify, please see the following :

  • E-Verify Poster (English and Spanish) [PDF]
  • Right to Work Poster (English) [PDF]
  • Right to Work Poster (Spanish) [PDF]
  • Compliance :

    Employees may be required to report violations of law under Title IX and the Jeanne Clery Disclosure of Campus Security Policy and Crime Statistics Act (Clery Act). If this position is identified a Campus Security Authority (Clery Act), you will be notified and provided resources for reporting. Responsible employees under Title IX are defined and outlined in HOP-3031 .

    The Clery Act requires all prospective employees be notified of the availability of the Annual Security and Fire Safety report. You may access the most recent report here or obtain a copy at University Compliance Services, 1616 Guadalupe Street, UTA 2.206, Austin, Texas 78701.

    Create a job alert for this search

    Lead Data Engineer • Austin, TX, United States

    Related jobs
    Lead Data Scientist - Director Level (IC)

    Lead Data Scientist - Director Level (IC)

    Visa • Austin, TX, United States
    Full-time
    Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more t...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    Interactive Resources - iR • Austin, TX, United States
    Full-time
    We are seeking a highly skilled.Azure cloud technologies, and Lakehouse architectures.This role is ideal for someone who thrives in dynamic environments, enjoys solving complex data challenges, and...Show more
    Last updated: 15 hours ago • Promoted • New!
    Data Engineer (Middle) ID43228

    Data Engineer (Middle) ID43228

    AgileEngine • Austin, TX, us
    Full-time
    Quick Apply
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Show more
    Last updated: 18 days ago
    Principal Data Engineer - AI Systems Performance Engineering

    Principal Data Engineer - AI Systems Performance Engineering

    Visa • Austin, Texas, United States
    Full-time
    Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more t...Show more
    Last updated: 30+ days ago • Promoted
    Sr. ML Engineer, AI Cloud

    Sr. ML Engineer, AI Cloud

    Tenstorrent • Austin, Texas, United States
    Full-time +1
    Tenstorrent is leading the industry on cutting-edge AI technology, revolutionizing performance expectations, ease of use, and cost efficiency. With AI redefining the computing paradigm, solutions mu...Show more
    Last updated: 30+ days ago • Promoted
    Service Engineer (27775)

    Service Engineer (27775)

    Supermicro • Austin, TX, United States
    Full-time
    Supermicro is a Top Tier provider of advanced server, storage, and networking solutions for Data Center, Cloud Computing, Enterprise IT, Hadoop / Big Data, Hyperscale, HPC and IoT / Embedded customers...Show more
    Last updated: 30+ days ago • Promoted
    Principal Product Manager - Director Cloudera Data Warehouse (Hive / Impala)

    Principal Product Manager - Director Cloudera Data Warehouse (Hive / Impala)

    Cloudera • Austin, TX, United States
    Full-time
    At Cloudera, we empower people to transform complex data into clear and actionable insights.With as much data under management as the hyperscalers, we're the preferred data partner for the top comp...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    University of Texas at Austin • Austin, TX, United States
    Full-time +1
    Enterprise Technology - Data to Insights (D2I).Expected to Continue Until Dec 19, 2026.This is a fixed term position that is expected to continue for a 1-year limited term from start date with a po...Show more
    Last updated: 15 hours ago • Promoted • New!
    Sr. Data Engineer

    Sr. Data Engineer

    Visa • Austin, TX, United States
    Full-time
    Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more t...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Luna Data Solutions, Inc. • Austin, TX, United States
    Full-time
    This will be a multi-year contract worked.The ideal candidate brings deep technical expertise in data architecture, ETL / ELT engineering, data integration, and governance, along with hands-on experi...Show more
    Last updated: 1 day ago • Promoted
    Data Engineer

    Data Engineer

    Intellectt Inc • Austin, TX, United States
    Full-time
    Data Engineering & Analytics Consultant.Cupertino, CA / Austin, TX / Seattle, WA.I am looking for EX-apple employee candidates. Data engineering, Advance proficiency in Python, SQL with 4-5 years ex...Show more
    Last updated: 1 day ago • Promoted
    Data Engineer (Austin)

    Data Engineer (Austin)

    Intellectt Inc • Austin, TX, US
    Part-time
    Data Engineering & Analytics Consultant.Cupertino, CA / Austin, TX / Seattle, WA.I am looking for EX-apple employee candidates. Data engineering, Advance proficiency in Python, SQL with 4-5 years ex...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Plum Inc • Austin, TX, US
    Remote
    Full-time
    Quick Apply
    PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...Show more
    Last updated: 30+ days ago
    Senior Release Engineer

    Senior Release Engineer

    Cloudera • Austin, TX, United States
    Full-time
    At Cloudera, we empower people to transform complex data into clear and actionable insights.With as much data under management as the hyperscalers, we're the preferred data partner for the top comp...Show more
    Last updated: 12 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    University of Texas at Austin • Austin, TX, United States
    Full-time +1
    Enterprise Technology - Data to Insights (D2I).Expected to Continue Until Dec 19, 2026.This is a fixed term position that is expected to continue for a 1-year limited term from start date with a po...Show more
    Last updated: 30+ days ago • Promoted
    Cloud Data Engineer

    Cloud Data Engineer

    AlphaPoint • Austin, TX, US
    Full-time
    Quick Apply
    About Us AlphaPoint’s AI Labs’ team of engineers and AI scientists is solving complex business problems by bridging the gap between transformative breakthroughs in AI technology and inc...Show more
    Last updated: 30+ days ago
    Senior Staff Engineer

    Senior Staff Engineer

    Cloudera • Austin, TX, United States
    Full-time
    At Cloudera, we empower people to transform complex data into clear and actionable insights.With as much data under management as the hyperscalers, we're the preferred data partner for the top comp...Show more
    Last updated: 9 days ago • Promoted
    Sr. Sales Director (25848)

    Sr. Sales Director (25848)

    Supermicro • Austin, TX, United States
    Full-time
    Supermicro is a Top Tier provider of advanced server, storage, and networking solutions for Data Center, Cloud Computing, Enterprise IT, Hadoop / Big Data, Hyperscale, HPC and IoT / Embedded customers...Show more
    Last updated: 30+ days ago • Promoted