AI-ML Quality Engineerinnovitusa • Des Moines, IA, Iowa, USA

AI-ML Quality Engineer

innovitusa • Des Moines, IA, Iowa, USA

9 hours ago

Job type

Full-time

Job description

Hiring : W2 Candidates Only

🛂 Visa : Open to any visa type with valid work authorization in the USA

Job Overview

We are seeking a highly skilled Senior AI-ML LLM Quality Engineer with strong expertise in testing and validating large language models (LLMs) and generative AI products. The ideal candidate will have hands-on experience in Python scripting, automation frameworks, and evaluation of LLMs within enterprise environments. This role involves building test strategies, executing model performance validations, and guiding customers on automation strategies.

Key Responsibilities

Support testing and validation of Large Language Model (LLM)-powered applications.
Design and implement test strategies, evaluation workflows, and automation frameworks for generative AI systems.
Perform model performance validation across diverse generative AI use cases.
Collaborate with cross-functional teams to ensure reliable, transparent, and scalable AI solutions.
Guide customers on automation strategy, relevant tools, and best practices.

Must-Have Skills

Strong experience in Python scripting, REST APIs, YAML.

Hands-on experience with testing Generative AI / ML products and evaluating LLMs in enterprise environments.

Experience with LLM Testing Tools (e.g., LangSmith, Promptfoo).

Strong understanding of LLM behavior and evaluation workflows.

Proficiency with PyTest, Selenium, or similar test automation frameworks.

Strong experience with testing automation and ability to advise customers on relevant technologies.

Nice-to-Have Skills

Experience with advanced testing frameworks.

Experience testing RAG pipelines and LLM agent systems.

Familiarity with LangChain, LlamaIndex, or Haystack.

Knowledge of AI / ML model evaluation metrics.

Experience with Red Teaming (preferred but not mandatory).

Familiarity with AWS cloud platforms and MLOps tooling (e.g., MLflow).

Ideal Candidate Profile

6 years of relevant AI / ML and software testing experience.

Deep understanding of LLM testing methodologies and automation strategies.

Strong problem-solving and communication skills to collaborate effectively with product and engineering teams.

Passion for ensuring AI model quality, transparency, and reliability in real-world enterprise applications.

Create a job alert for this search

Quality Engineer • Des Moines, IA, Iowa, USA

Related jobs

Mobile Software Quality Engineer

Robert Half • Des Moines, IA, US

Full-time

Position : SENIOR SOFTWARE QUALITY ENGINEER - Shape the future of a Mobile-First Platform.Salary : UP TO $125K BASE + BONUS + EXCEPTIONAL BENEFITS. For immediate and confidential consideration, please...Show more

Last updated: 11 days ago • Promoted

Physician / Pulmonology / Iowa / Locum Tenens / Locums / Pulmonology / Job / Pennsylvania Job

MD Staff, LLC • Altoona, IA, US

Full-time

Looking for a BE or BC Pulmonologist for locums coverage in Pennsylvania.Only providers with Active PA Licenses.Patient census : Up to 20 patients at times. Procedures : Thoracenthesis, Bronchoscopies...Show more

Last updated: 30+ days ago • Promoted

AI / ML Developer

DataAnnotation • Nevada, IA, US

Full-time +1

We are looking for an AI / ML Developer to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each...Show more

Last updated: 19 days ago • Promoted

Corporate Controls Engineer

CyberCoders • Boone, IA, US

Full-time

Title : Senior Controls Engineer.Salary : $110,000 - $130,000 .Founded in 1917, we are a leading producer of dairy products, serving major retailers and foodservice clients across North America....Show more

Last updated: 13 days ago • Promoted

Artificial Intelligence (AI) System Analyst

Walsh Door & Security • Des Moines, IA, US

Full-time

JOB OPENING – Artificial Intelligence (AI) System Analyst.Join a company with 160 years of excellence and a bright future ahead!. At Walsh Door + Security, we’re proud to be a family-own...Show more

Last updated: 8 days ago • Promoted

Market Research Contributor

Prime Insights • Boone, IA, US

Full-time

Join thousands of members already earning with top-paying surveys and offers.Get started today and enjoy competitive rewards, fast payouts with no waiting periods, and the flexibility to participat...Show more

Last updated: 1 day ago • Promoted

Fresh Cut QC Coordinator

Loffredo Fresh Foods • Norwalk, IA, US

Full-time

Full time Fresh Cut Quality Control Coordinator.We are a local, family-owned food and produce distributor that has proudly served the Midwest region for over 130 years. We distribute to 11 different...Show more

Last updated: 30+ days ago • Promoted

Lead Quality Engineer - (POS Modernization)

Kitestring • Des Moines, IA, US

Full-time

Kitestring is a 20+-million-dollar company with 100+ associates and growing.We are a 25 year old start up in Northwest Arkansas. We provide technical expertise to various clients delivering innovati...Show more

Last updated: 7 days ago • Promoted

Quality Inspector @ Ames, Iowa (1099 Contractor)

Pro QC International North America • Ames, IA, US

Full-time

Supervise and audit truck loading / unloading of 2040 wooden book carts per truck.Verify compliance with Googles packing / handling instructions at pickup library sites. Observe loading process, documen...Show more

Last updated: 30+ days ago • Promoted

Otolaryngology Opportunity in Ames, Iowa

First Choice Inc • Ames, IA, US

Full-time +1

Physician-Led Medicine in IowaMcFarland Clinic is seeking a BE / BC Otolaryngologist to join our vibrant and collegial team in Ames, Iowa within Iowas largest physician-owned, multi-specialty clinic....Show more

Last updated: 30+ days ago • Promoted

Meat / Produce Team Associate

Walmart • Altoona, IA, US

Full-time +1

The actual hourly rate will equal or exceed the required minimum wage applicable to the job location.Additional compensation includes annual or quarterly performance incentives.Additional compensat...Show more

Last updated: 16 days ago • Promoted

Data Engineer

DataAnnotation • Nevada, IA, US

Full-time +1

We are looking for a Data Engineer to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each mo...Show more

Last updated: 19 days ago • Promoted

Physician / Neurology / Iowa / Locum Tenens / Locums / Neurology / Job / Pennsylvania Job

MD Staff, LLC • Altoona, IA, US

Full-time

Neurologist needed for a 24hr call coverage in Pennsylvania from October to ongoing.Total patients seen are 10-15 daily.Great pay with benefits! If you are interested in hearing more about this opp...Show more

Last updated: 30+ days ago • Promoted

Mobile SW Quality Engineer

Robert Half • Des Moines, IA, US

Full-time

Last updated: 11 days ago • Promoted

Emergency Medicine Physician Needed for Locum Tenens Coverage at Facility in Jefferson, Iowa

LocumTenens.com • Maxwell, IA, US

Full-time

Job Reference Id : ORD-203628-MD-IA.Dates Needed : December - ongoing.Shift Type : Day Shift; Night Shift.Board Certification Required : Yes. A facility is seeking an Emergency Medicine Physician fo...Show more

Last updated: 1 day ago • Promoted

Quality Control Technician

Continental Manufacturing • Madrid, IA, US

Full-time

The Lab Quality Control Technician ensures product quality and regulatory compliance by performing testing, inspections, and documentation throughout the manufacturing process.This role includes ex...Show more

Last updated: 1 day ago • Promoted

Skilled Welder / Robotic Welder

GRIP-TITE MANUFACTURING CO,LLC • Winterset, IA, US

Full-time

Skilled Welder / Robotic Welder(s).OT as required based on business need.This will include Friday work as needed.We are seeking skilled and detail-oriented Welders / Robotic Welders to join our dyn...Show more

Last updated: 30+ days ago • Promoted

Manufacturing Quality Engineer

Reach Hire Talent Solutions • Ames, IA, US

Full-time +1

Reach Hire Talent Solutions provided pay range.This range is provided by Reach Hire Talent Solutions.Your actual pay will be based on your skills and experience — talk with your recruiter to learn ...Show more

Last updated: 5 hours ago • Promoted • New!