Talent.com
AI Agent Evaluation Analyst

AI Agent Evaluation Analyst

VirtualVocationsVirginia Beach, Virginia, United States
1 day ago
Job type
  • Full-time
Job description

A company is looking for an AI Agent Evaluation Analyst.

Key Responsibilities

Review evaluation tasks and scenarios for logic, completeness, and realism

Identify inconsistencies, missing assumptions, or unclear decision points

Help define clear expected behaviors for AI agents

Required Qualifications

Excellent analytical thinking regarding complex systems and logical implications

Familiarity with structured data formats like JSON / YAML

Experience with policy evaluation, logic puzzles, or structured scenario design

Background in consulting, academia, or research

Some understanding of scoring or evaluation in agent testing

Create a job alert for this search

Ai Analyst • Virginia Beach, Virginia, United States

Related jobs
  • Promoted
TECHNICAL ANALYST I

TECHNICAL ANALYST I

VSolvitChesapeake, VA, US
Full-time
A Technical Analyst analyzes technical information and aligns it to the business strategies and procedures in meeting project goals. They monitor technical system errors and provide immediate resolu...Show moreLast updated: 5 days ago
  • Promoted
  • New!
Senior Machine Learning Engineer

Senior Machine Learning Engineer

Capital OneNorfolk, VA, US
Full-time +1
Senior Machine Learning Engineer As a Capital One Machine Learning Engineer (MLE), you'll be part of an Agile team dedicated to productionizing machine learning applications and systems at scale.Yo...Show moreLast updated: 9 hours ago
  • Promoted
DATA AND AI SENIOR SOFTWARE DEVELOPER

DATA AND AI SENIOR SOFTWARE DEVELOPER

DEFTEC CorporationChesapeake, VA, US
Full-time
DATA AND AI SENIOR SOFTWARE DEVELOPER.DEFTEC delivers mission-critical solutions through skillfully delivered services and innovative products. We are inspired by our clients' critical missions ...Show moreLast updated: 2 days ago
Data Scientist / Software Engineer

Data Scientist / Software Engineer

G2 Ops, Inc.Virginia Beach, VA, US
Full-time
Quick Apply
Virginia Beach, VA at our wonderful G2 Ops office.In person, some remote opportunity and / or flexible working hours, not a fully remote position. August / September 2025 Salary Range : $130-$150K p...Show moreLast updated: 30+ days ago
STI is hiring : AI Developer UI Strategy in Virginia Beach

STI is hiring : AI Developer UI Strategy in Virginia Beach

MediabistroVirginia Beach, VA, United States
Full-time
Contractor will need to come to Richmond quarterly.The Virginia Department of Transportation's Information Technology Division is seeking a highly skilled AI Developer UI Strategy mainly crafting a...Show moreLast updated: 7 days ago
1.2 Campaign Data Manager

1.2 Campaign Data Manager

Kingston StrategyEastville, VA, US
Temporary
Quick Apply
Campaign Data Manager Kingston Strategy Location : .Suffolk / Virginia Beach / Eastville, VA Job Type : Temporary, Contract Experience Level : 1-2 Cycles preferred, not required Compensation : Sal...Show moreLast updated: 30+ days ago
Database Management Analyst lll

Database Management Analyst lll

ENGINEERING SERVICES NETWORK, Inc.Chesapeake, VA, USA
Full-time
Quick Apply
Founded in 1995, ESN is a Small Disadvantaged Business (SDB), Service-Disabled Veteran-Owned Small Business (SDVOSB) and Veteran Owned Small Business (VOSB). ESN delivers trusted solutions that supp...Show moreLast updated: 30+ days ago
  • Promoted
Data Analytics Manger-SQL to Big Query

Data Analytics Manger-SQL to Big Query

Family DollarChesapeake, VA, US
Full-time
We cannot provide sponsorship at this time.Data Analytics Manager-SQL to Big Query.At Family Dollar, we strive for exceptional collaboration between talented individuals who exhibit our values and ...Show moreLast updated: 11 days ago
  • Promoted
  • New!
Distinguished AI Engineer

Distinguished AI Engineer

Capital OneNewport News, VA, US
Full-time +1
Distinguished AI Engineer Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machin...Show moreLast updated: 22 hours ago
  • Promoted
Data Analytics Manger-SQL to Big Query (Chesapeake)

Data Analytics Manger-SQL to Big Query (Chesapeake)

Family DollarChesapeake, VA, US
Part-time
We cannot provide sponsorship at this time.Data Architect-SQL to Big Query.At Family Dollar, we strive for exceptional collaboration between talented individuals who exhibit our values and execute ...Show moreLast updated: 7 days ago
  • Promoted
  • New!
SCADA / PLC / HMI Programmer -5+ yrs Required

SCADA / PLC / HMI Programmer -5+ yrs Required

Systems East, Inc.Seaford, Virginia, US
Full-time
Control Systems Integration firm is currently seeking full-time SCADA / PLC / HMI Programmers with a minimum of 5+ years of experience using Ignition SCADA, Allen Bradley FactoryTalk, VT SCADA, Schn...Show moreLast updated: 1 hour ago
CYBERSPACE CONCEPT DEVELOPER AND VALIDATOR

CYBERSPACE CONCEPT DEVELOPER AND VALIDATOR

DEFTEC CorporationNorfolk, VA, USA
Full-time
Quick Apply
CYBERSPACE CONCEPT DEVELOPER AND VALIDATOR.DEFTEC delivers mission-critical solutions through skillfully delivered services and innovative products. We are inspired by our clients' critical missions...Show moreLast updated: 2 days ago
  • Promoted
  • New!
Lead Machine Learning Engineer (Intelligent Foundations & Experiences)

Lead Machine Learning Engineer (Intelligent Foundations & Experiences)

Capital OneNewport News, VA, US
Full-time +1
Lead Machine Learning Engineer (Intelligent Foundations & Experiences) As a Capital One Machine Learning Engineer (MLE), you'll be part of an Agile team dedicated to productionizing machine learnin...Show moreLast updated: 22 hours ago
Cyberspace Concept Developer & Validator

Cyberspace Concept Developer & Validator

Ironclad Defense WorksNorfolk, VA, US
Full-time
Quick Apply
Cyberspace Concept Developer & Validator (LC 32) – Onsite at NATO ACT Norfolk Location : .HQ SACT, Norfolk, VA Position Type : Full-time, Onsite Clearance Requirement : Active NATO or Nationa...Show moreLast updated: 17 days ago
  • Promoted
  • New!
Senior Data Engineer

Senior Data Engineer

Capital OneNorfolk, VA, US
Full-time +1
Senior Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery en...Show moreLast updated: 22 hours ago
Cyberspace Concept Developer and Validator

Cyberspace Concept Developer and Validator

SpektrumNorfolk, Virginia, United States
Full-time
Quick Apply
Spektrum have a wide range of exciting opportunities in several global locations.We are always looking to add great new talent to our team and look forward to hearing from you.Spektrum supports ape...Show moreLast updated: 12 days ago
  • Promoted
  • New!
Lead Machine Learning Engineer - ML / AI

Lead Machine Learning Engineer - ML / AI

Capital OneNorfolk, VA, US
Full-time +1
Lead Machine Learning Engineer - ML / AI At Capital One, we are changing banking for good by creating responsible and reliable AI-powered systems. Our investments in technology infrastructure and worl...Show moreLast updated: 22 hours ago
  • Promoted
  • New!
Senior Lead Data Engineer

Senior Lead Data Engineer

Capital OneNewport News, VA, US
Full-time +1
Senior Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delive...Show moreLast updated: 22 hours ago
  • Promoted
DATA SCIENCE AND AI ENGINEER

DATA SCIENCE AND AI ENGINEER

DEFTEC CorporationChesapeake, VA, US
Full-time
DEFTEC delivers mission-critical solutions through skillfully delivered services and innovative products.We are inspired by our clients' critical missions and driven to provide the most effecti...Show moreLast updated: 2 days ago
  • Promoted
DATA AND AI FULL STACK DEVELOPER

DATA AND AI FULL STACK DEVELOPER

DEFTEC CorporationChesapeake, VA, US
Full-time
DATA AND AI FULL STACK DEVELOPER.DEFTEC delivers mission-critical solutions through skillfully delivered services and innovative products. We are inspired by our clients' critical missions and d...Show moreLast updated: 2 days ago