Talent.com
AI Agent Evaluation Analyst

AI Agent Evaluation Analyst

VirtualVocationsLincolnwood, Illinois, United States
17 hours ago
Job type
  • Full-time
Job description

A company is looking for an AI Agent Evaluation Analyst.

Key Responsibilities

Review evaluation tasks and scenarios for logic, completeness, and realism

Identify inconsistencies, missing assumptions, or unclear decision points

Help define clear expected behaviors for AI agents

Required Qualifications

Excellent analytical thinking regarding complex systems and logical implications

Familiarity with structured data formats like JSON / YAML

Experience with policy evaluation, logic puzzles, or structured scenario design

Background in consulting, academia, or research

Some understanding of scoring or evaluation in agent testing

Create a job alert for this search

Ai Analyst • Lincolnwood, Illinois, United States

Related jobs
  • Promoted
Product Manager, Public Sector Platform & AI

Product Manager, Public Sector Platform & AI

RelativityChicago, IL, United States
Full-time
The Product Management team is looking for a Product Manager with a strong background in product lifecycle and DevOps / Agile practices as well as strong communication and collaboration skills.You wi...Show moreLast updated: 19 days ago
  • Promoted
Generative AI Specialist

Generative AI Specialist

VirtualVocationsLincolnwood, Illinois, United States
Full-time
A company is looking for a Generative AI Specialist - Humanities (English and Slovak).Key Responsibilities Evaluate and rate the performance of AI models based on output Annotate and classify co...Show moreLast updated: 30+ days ago
  • Promoted
Remote Financial Analyst - AI Trainer

Remote Financial Analyst - AI Trainer

Data AnnotationElmhurst, Illinois
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
Remote Financial Planner - AI Trainer

Remote Financial Planner - AI Trainer

Data AnnotationOak Lawn, Illinois
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
Remote AI Writing Evaluator

Remote AI Writing Evaluator

OutlierChicago, IL, United States
Remote
Full-time
Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...Show moreLast updated: 5 days ago
  • Promoted
  • New!
Search Analyst

Search Analyst

VirtualVocationsNaperville, Illinois, United States
Full-time
A company is looking for a Search Analyst to support SEO and SEM campaigns.Key Responsibilities Assist with planning, execution, optimization, and reporting of SEM campaigns Support organic sear...Show moreLast updated: 9 hours ago
  • Promoted
  • New!
AI Content Editor

AI Content Editor

VirtualVocationsGary, Indiana, United States
Full-time
A company is looking for an AI Content Editor to join their SEO team in a remote, full-time capacity.Key Responsibilities Proofread and fact-check AI-assisted content for accuracy, clarity, and c...Show moreLast updated: 15 hours ago
  • Promoted
  • New!
AI Evaluation Analyst

AI Evaluation Analyst

VirtualVocationsLincolnwood, Illinois, United States
Full-time
A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...Show moreLast updated: 17 hours ago
  • New!
ICONMA is hiring : AI Multimedia Generation Product Evaluator in Chicago

ICONMA is hiring : AI Multimedia Generation Product Evaluator in Chicago

MediabistroChicago, IL, United States
Full-time
Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location. The Multimedia Generation team focuses on fine tuning an...Show moreLast updated: 8 hours ago
  • Promoted
Remote Senior Financial Analyst - AI Trainer

Remote Senior Financial Analyst - AI Trainer

Data AnnotationAddison, Illinois
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
AI Content Writer - Part Time Work From Home

AI Content Writer - Part Time Work From Home

OutlierJoliet, IL, United States
Remote
Part-time
Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...Show moreLast updated: 5 days ago
  • Promoted
AI Trainer

AI Trainer

VirtualVocationsGary, Indiana, United States
Full-time
A company is looking for an AI Trainer / Senior Prompt Evaluation.Key Responsibilities Conduct thorough reviews of AI model dialogues with users to assess quality and relevance Identify growth a...Show moreLast updated: 21 days ago
  • Promoted
Remote Finance Advisor - AI Trainer

Remote Finance Advisor - AI Trainer

Data AnnotationHammond, Indiana
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
Analyst, Research (Chicago / Hybrid)

Analyst, Research (Chicago / Hybrid)

Feeding AmericaChicago, IL, United States
Full-time
Imagine a place where your talent can make a meaningful difference in peoples' lives.Working at Feeding America is a uniquely rewarding experience in which our employees work together as vital part...Show moreLast updated: 5 days ago
  • Promoted
Board Certified Behavioral Analyst

Board Certified Behavioral Analyst

Action Behavior CentersRomeoville, IL, United States
Full-time
Join Action Behavior Centers as a BCBA.And we commit to a response within 24 hours.Ability to earn more through additional performance-based bonuses. Bonuses kick in at just 26 hours of treatment de...Show moreLast updated: 30+ days ago
  • Promoted
English AI Writer - Part Time Work From Home

English AI Writer - Part Time Work From Home

OutlierJoliet, IL, United States
Remote
Part-time
Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...Show moreLast updated: 5 days ago
  • Promoted
Remote Commercial Banking Analyst - AI Trainer

Remote Commercial Banking Analyst - AI Trainer

Data AnnotationCarol Stream, Illinois
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
AI Engineer

AI Engineer

VirtualVocationsLincolnwood, Illinois, United States
Full-time
A company is looking for an AI Engineer (FinTech Workflow Automation).Key Responsibilities Design, build, and deploy AI models into production Develop and maintain backend Python services for in...Show moreLast updated: 30+ days ago