AI Agent Evaluation AnalystVirtualVocations • Largo, Florida, United States
No longer accepting applications
AI Agent Evaluation Analyst
VirtualVocations • Largo, Florida, United States
19 days ago
Job type
- Full-time
Job description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats like JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
Create a job alert for this search
Ai Analyst • Largo, Florida, United States
Related jobs
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for an AI Solutions Analyst to support the design and adoption of AI workstreams from an administrative perspective.
Key Responsibilities Partner with teams to identify high-v...Show more A company is looking for an AI Agent Developer.Key Responsibilities Architect and implement AI-powered agents using Large Language Models (LLMs) Connect agents to internal systems and databases,...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more 
WHEN YOU WORK FOR US, WE WORK FOR YOU\n.With Titan Medical, you gain access to thousands of travel nursing and allied health jobs across the country.
You also get unmatched service.From the moment y...Show more
Host Healthcare is seeking an experienced Cath Lab Tech for an exciting Travel Allied job in Wimauma, FL.Shift : 10 hr days Start Date : 11 / 18 / 2025 Duration : 13 weeks Pay : $2080.At Host Healthcare, w...Show more A company is looking for an AI Multimedia Generation Product Evaluator.Key Responsibilities Set up Halo tasks and run model inference Audit annotations from vendors and manage evaluation dataset...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for an Applied AI Engineer to develop and deploy AI models and hybrid AI frameworks.Key Responsibilities Develop and deploy AI models and hybrid AI frameworks Conduct resear...Show more A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...Show more 
AI Agent Developer (LLM Specialist).The AI Agent Developer is responsible for designing, building, and deploying intelligent autonomous or semi-autonomous agents powered by Large Language Models (L...Show more A company is looking for a Software Engineer II to evaluate AI multimedia generation products for images and videos.Key Responsibilities Set up Halo tasks and run model inference Audit annotatio...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for an AI Evaluator / Annotator (Remote- freelance, 100+ openings).Key Responsibilities Evaluate outputs generated by large language models (LLMs) across multiple modalities ...Show more A company is looking for a Multimodal AI Evaluation Analyst.Key Responsibilities Evaluate and score AI-generated outputs across various modalities Assess quality, correctness, coherence, style, ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for an Agent Evaluation Analyst.Key Responsibilities Take ownership of delivery team projects, ensuring data quality and timely delivery Lead a cross-functional team to keep...Show more
Remote Equity Research Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • St. Petersburg, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
AI Solutions Analyst
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted
AI Agent Developer
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted
Remote Financial Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Pinellas Park, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
Travel - CT Technologist
Titan Medical Group • Sun City, FL, US
Full-time
Last updated: 1 hour ago • Promoted • New!
Travel Cath Lab Tech - $2080.89 / Week
Host Healthcare • Wimauma, FL, US
Full-time
Last updated: 3 days ago • Promoted
AI Multimedia Evaluator
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 3 days ago • Promoted
Remote Investment Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • St. Petersburg, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
Applied AI Engineer
VirtualVocations • Saint Petersburg, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted
AI Agent Evaluation Analyst
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 19 days ago • Promoted
AI Agent Developer
ClearlyAgile • Tampa, FL, US
Full-time
Last updated: 1 day ago • Promoted
AI Multimedia Generation Evaluator
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 2 days ago • Promoted
Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Tampa, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
AI Evaluation Analyst
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 19 days ago • Promoted
Multimodal AI Evaluation Analyst
VirtualVocations • Tampa, Florida, United States
Full-time
Last updated: 1 day ago • Promoted
Remote Corporate Development Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • Largo, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
Remote FP&A Manager – AI Trainer ($50-$60 / hour)
Data Annotation • Pinellas Park, Florida
Remote
Full-time +1
Last updated: 10 days ago • Promoted
AI Data Project Manager
VirtualVocations • Saint Petersburg, Florida, United States
Full-time
Last updated: 2 days ago • Promoted