Talent.com
AI Evaluation Analyst

AI Evaluation Analyst

VirtualVocationsNewark, Delaware, United States
3 hours ago
Job type
  • Full-time
Job description

A company is looking for an AI Agent Evaluation Analyst.

Key Responsibilities

Review evaluation tasks and scenarios for logic, completeness, and realism

Identify inconsistencies, missing assumptions, or unclear decision points

Help define clear expected behaviors (gold standards) for AI agents

Required Qualifications

Excellent analytical thinking regarding complex systems and logical implications

Familiarity with structured data formats, such as JSON / YAML

Experience with policy evaluation, logic puzzles, or structured scenario design

Background in consulting, academia, or research

Some understanding of scoring or evaluation in agent testing

Create a job alert for this search

Ai Analyst • Newark, Delaware, United States

Related jobs
  • Promoted
Data Analytics Manager

Data Analytics Manager

Incyte CorporationChadds Ford, PA, United States
Full-time
Incyte is a biopharmaceutical company focused on the discovery, development, and commercialization of novel medicines to meet serious unmet medical needs in oncology and inflammation and autoimmuni...Show moreLast updated: 30+ days ago
  • Promoted
Models Mission Control, Analyst

Models Mission Control, Analyst

Chatham FinancialKennett Square, PA, United States
Full-time
We don't simply hire employees.When you work at Chatham, we empower you - offering professional development opportunities to help you grow in your career, no matter if you've been here for five mon...Show moreLast updated: 9 days ago
Open Systems Technologies is hiring : AML SAR Writer in New Castle County

Open Systems Technologies is hiring : AML SAR Writer in New Castle County

MediabistroNew Castle County, DE, United States
Full-time
Location : New Castle, DE - 3 days onsite / week.SAR writing exp is required and crucial.They do not like hoppy resumes.The AML Analysts within the CRIU review system-generated and manual cases for ac...Show moreLast updated: 30+ days ago
  • Promoted
AI Workflow Architect Intern

AI Workflow Architect Intern

AmerisourceBergen Corporation (Cencora)Conshohocken, PA, United States
Full-time
Our team members are at the heart of everything we do.At Cencora, we are united in our responsibility to create healthier futures, and every person here is essential to us being able to deliver on ...Show moreLast updated: 12 days ago
  • Promoted
Party Reference Data Analyst

Party Reference Data Analyst

Veterans StaffingNewark, DE, US
Full-time
Are you ready to make an impact in a dynamic, data-driven environment? As a Reference Data Analyst, you will play a key role in managing and maintaining client reference data across multiple busine...Show moreLast updated: 5 days ago
Kore.ai Business Analyst

Kore.ai Business Analyst

iSoftTek Solutions IncMalvern, PA, US
Full-time
Quick Apply
Malvern, PA (hybrid – 3 days / week from office).As a Business Analyst, you will be part of the team building modern Contact Center applications for our leading Financial Services customers.You will ...Show moreLast updated: 30+ days ago
Freelance Luxury Brand Evaluator Automotive Project - Pittsburgh & Eastern PA

Freelance Luxury Brand Evaluator Automotive Project - Pittsburgh & Eastern PA

CXGW CNSHOHOCKEN, PA, US
Full-time
Quick Apply
Are you a luxury automobile enthusiast who appreciates the finer details of high-end vehicles? If the answer is yes, we are looking for you!. As a Luxury Brand Evaluator, you will step into the worl...Show moreLast updated: 12 days ago
  • Promoted
Board Certified Behavior Analyst

Board Certified Behavior Analyst

Helping Hands Family - Autism ServicesChestnut Hill, PA, US
Full-time
Built by clinicians for clinicians, with a mission-driven, outcome-focused culture.Supportive team of Clinical Directors, peer BCBAs, and five Regional Directors of Quality who provide hands-on men...Show moreLast updated: 8 days ago
  • Promoted
AI Readiness and Enablement Intern

AI Readiness and Enablement Intern

AmerisourceBergen Corporation (Cencora)Conshohocken, PA, United States
Full-time
Our team members are at the heart of everything we do.At Cencora, we are united in our responsibility to create healthier futures, and every person here is essential to us being able to deliver on ...Show moreLast updated: 12 days ago
  • Promoted
Remote Senior Financial Analyst - AI Trainer

Remote Senior Financial Analyst - AI Trainer

Data AnnotationWilmington, Delaware
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
  • Promoted
Travel Board Certified Behavioral Analyst (BCBA) - School in Wilmington, DE

Travel Board Certified Behavioral Analyst (BCBA) - School in Wilmington, DE

AlliedTravelCareersNewark, Delaware, US
Full-time
AlliedTravelCareers is working with Aequor to find a qualified BCBA - School in Wilmington, Delaware, 19805!.Board Certified Behavior Analyst (BCBA). With Aequor, you can enjoy the freedom to advanc...Show moreLast updated: 17 days ago
  • Promoted
Analyst, Investment Research

Analyst, Investment Research

PathstoneBroomall, PA, US
Full-time
Pathstone is a dynamic, rapidly growing, partner-owned wealth management firm that serves ultra-high net worth families, single family offices, foundations and endowments.Our comprehensive range of...Show moreLast updated: 8 days ago
  • Promoted
  • New!
Principal Associate, Ontology and Data Modeling- Retail Bank

Principal Associate, Ontology and Data Modeling- Retail Bank

Capital OneWilmington, DE, US
Full-time +1
Principal Associate, Ontology and Data Modeling- Retail Bank.The role of the Principal Associate of Ontology and Data Modeling is to develop, implement, and maintain enterprise ontologies in suppor...Show moreLast updated: 4 hours ago
  • Promoted
Remote Financial Analyst - AI Trainer

Remote Financial Analyst - AI Trainer

Data AnnotationWilmington, Delaware
Remote
Full-time +1
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...Show moreLast updated: 30+ days ago
Freelance Luxury Brand Evaluator Automotive Project - United States

Freelance Luxury Brand Evaluator Automotive Project - United States

CXGNewtown Square, PA, US
Full-time
Quick Apply
Are you a luxury automobile enthusiast who appreciates the finer details of high-end vehicles? If the answer is yes, we are looking for you!. As a Luxury Brand Evaluator, you will step into the worl...Show moreLast updated: 12 days ago
  • Promoted
(Sr. / Lead) Analyst I (II)

(Sr. / Lead) Analyst I (II)

PJM InterconnectionAudubon, PA, US
Full-time
Flexible Work Arrangement - Hybrid.The Operational Excellence department provides oversight and management for PJM's Operational Excellence (OpEx) program and Internal Controls program.Its primary ...Show moreLast updated: 30+ days ago
  • Promoted
Strategy Analyst (Hybrid)

Strategy Analyst (Hybrid)

The Bancorp Bank, N.A.Wilmington, DE, United States
Full-time
After the initial training period, this is a hybrid role, working onsite in our Wilmington, DE office.The analyst will manage strategic data modeling, forecasting, analysis, industry research, repo...Show moreLast updated: 30+ days ago
  • Promoted
NAM-US-AML Investigation Analyst-Analyst (USD)

NAM-US-AML Investigation Analyst-Analyst (USD)

StaffingNew Castle, DE, US
Full-time
Saydeli Manager : Jonathan King Location : New Castle, DE End Date : 1 / 14 / 2026 Rate : • • • / hr Role : AML SAR WRITER SAR Writing Exp Is Required. The AML Analysts within the CRIU review system-generated an...Show moreLast updated: 30+ days ago
  • Promoted
Business Solutions Analyst III - AI

Business Solutions Analyst III - AI

AmerisourceBergen Corporation (Cencora)Conshohocken, PA, United States
Full-time
Our team members are at the heart of everything we do.At Cencora, we are united in our responsibility to create healthier futures, and every person here is essential to us being able to deliver on ...Show moreLast updated: 12 days ago
  • Promoted
Market Analyst (Client Focused) - Hybrid (Berwyn, PA)

Market Analyst (Client Focused) - Hybrid (Berwyn, PA)

OneDigitalBerwyn, PA, US
Full-time
Employee Benefits is the core of our business and at our heart we are a people-focused organization.Every day, we help over 100,000 employers give their employees peace of mind by providing health,...Show moreLast updated: 30+ days ago