Talent.com
No longer accepting applications
AI Evaluation Analyst

AI Evaluation Analyst

VirtualVocationsToledo, Ohio, United States
2 days ago
Job type
  • Full-time
Job description

A company is looking for an AI Agent Evaluation Analyst.

Key Responsibilities

Review evaluation tasks and scenarios for logic, completeness, and realism

Identify inconsistencies, missing assumptions, or unclear decision points

Help define clear expected behaviors (gold standards) for AI agents

Required Qualifications

Excellent analytical thinking regarding complex systems and logical implications

Familiarity with structured data formats, such as JSON / YAML

Experience with policy evaluation, logic puzzles, or structured scenario design

Background in consulting, academia, or research

Some understanding of scoring or evaluation in agent testing

Create a job alert for this search

Ai Analyst • Toledo, Ohio, United States

Related jobs
  • Promoted
AI Solutions Specialist

AI Solutions Specialist

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for an AI Solutions Specialist to drive business innovation through artificial intelligence.Key Responsibilities Analyze complex datasets to uncover trends and actionable ins...Show moreLast updated: 30+ days ago
  • Promoted
AI Data Scientist

AI Data Scientist

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Data Scientist (AI Focus).Key Responsibilities Design and build natural language interaction capabilities using Python and Jupyter environments Act as a subject matter...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer with GCP Certification

Data Engineer with GCP Certification

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Data Engineer (Gen AI).Key Responsibilities Install and test Looker and Looker Studio extensions or API plugins Identify and deprecate unused Looker dashboards Enhanc...Show moreLast updated: 1 day ago
  • Promoted
AI Implementation Specialist

AI Implementation Specialist

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for an AI Implementation Specialist for Automated Portfolio Analysis and Reporting.Key Responsibilities Develop a GUI-based prototype for AI-powered document analysis and vis...Show moreLast updated: 30+ days ago
  • Promoted
Junior Data Engineer

Junior Data Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Business Intelligence : Jr Data Engineer.Key Responsibilities Assist the Analytics team in enhancing data reliability and accessibility for internal data units Extract ...Show moreLast updated: 30+ days ago
  • Promoted
  • New!
Data and Automation Lead

Data and Automation Lead

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Data and Automation Lead to support program management, automation engineering, data configuration, data analytics, and reporting activities. Key Responsibilities Provid...Show moreLast updated: 10 hours ago
  • Promoted
Business Intelligence Developer

Business Intelligence Developer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a BI Developer to transform data into actionable insights supporting strategic decision-making.Key Responsibilities Design, develop, deploy, and document interactive dash...Show moreLast updated: 30+ days ago
  • Promoted
Generative AI Scientist

Generative AI Scientist

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Generative AI Scientist.Key Responsibilities Deliver solutions to identify payment integrity issues and improve healthcare quality Develop exploratory data analysis ap...Show moreLast updated: 1 day ago
  • Promoted
Generative AI Specialist

Generative AI Specialist

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Generative AI Specialist - Humanities (English and Slovak).Key Responsibilities Evaluate and rate the performance of AI models based on output Annotate and classify co...Show moreLast updated: 30+ days ago
  • Promoted
Analytics Engineer

Analytics Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for an Analytics Engineer to support their operations team by leveraging data to understand user needs and improve customer experiences. Key Responsibilities Refine data, buil...Show moreLast updated: 30+ days ago
  • Promoted
  • New!
AI Developer Consultant

AI Developer Consultant

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Freelance Developer Consultant (Kotlin) - AI Trainer.Key Responsibilities Code generation and code review Training and evaluation of large language models Collaborati...Show moreLast updated: 10 hours ago
  • Promoted
Junior Machine Learning Engineer

Junior Machine Learning Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Junior Machine Learning Engineer to support the design, development, and deployment of machine learning models. Key Responsibilities Support the end-to-end lifecycle of ...Show moreLast updated: 30+ days ago
  • Promoted
Senior Business Intelligence Developer

Senior Business Intelligence Developer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Senior Business Intelligence Developer - Product & Platform Innovation.Key Responsibilities Collaborate with cross-functional teams to drive data enablement and support...Show moreLast updated: 30+ days ago
  • Promoted
Senior Big Data Engineer

Senior Big Data Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Senior Big Data Engineer to join their team.Key Responsibilities Support the transition from a monolithic application to loosely coupled microservices Ensure data is c...Show moreLast updated: 30+ days ago
  • Promoted
  • New!
EDI Analyst

EDI Analyst

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for an EDI Analyst responsible for internal EDI processes and implementation.Key Responsibilities Maintain knowledge of employee benefit systems and processes related to EDI ...Show moreLast updated: 12 hours ago
  • Promoted
Senior Analytics Engineer

Senior Analytics Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Senior Analytics Engineer.Key Responsibilities Design and maintain data models and transformation logic for analytics and reporting Own the semantic layer, managing sh...Show moreLast updated: 30+ days ago
  • Promoted
Deep Learning Software Engineer

Deep Learning Software Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Deep Learning Software Engineer, Inference and Model Optimization - New College Grad 2025.Key Responsibilities Train, develop, and deploy generative AI models using the...Show moreLast updated: 1 day ago
  • Promoted
AI Developer

AI Developer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for an AI Developer to advance the development of intelligent applications remotely.Key Responsibilities : Design and develop AI applications with OpenAI GPT models, ensuring ...Show moreLast updated: 30+ days ago
  • Promoted
Generative Machine Learning Engineer

Generative Machine Learning Engineer

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Generative Machine Learning Engineer.Key Responsibilities Develop and implement Deep Learning Pipelines using cutting-edge models Sustain the infrastructure for Deep L...Show moreLast updated: 30+ days ago
  • Promoted
Senior AI Consultant

Senior AI Consultant

VirtualVocationsToledo, Ohio, United States
Full-time
A company is looking for a Senior Consultant AI.Key Responsibilities Utilize Azure AI Services to build and optimize OpenAI solutions Develop and manage data ingestion and transformation pipelin...Show moreLast updated: 30+ days ago