AI Agent Evaluation AnalystVirtualVocations • Hollywood, Florida, United States
No longer accepting applications
AI Agent Evaluation Analyst
VirtualVocations • Hollywood, Florida, United States
20 days ago
Job type
- Full-time
Job description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats like JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
Create a job alert for this search
Ai Analyst • Hollywood, Florida, United States
Related jobs A company is looking for an AI Solutions Analyst to support the design and adoption of AI workstreams from an administrative perspective.
Key Responsibilities Partner with teams to identify high-v...Show more A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...Show more 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for an AI Multimedia Generation Product Evaluator.Key Responsibilities Set up Halo tasks and run model inference Audit annotations from vendors and manage evaluation dataset...Show more 
Job Description Are you a driven, customer-focused individual looking to build a successful career in real estate? Our local Weichert® franchised company is seeking enthusiastic Real Estate Agents ...Show more A company is looking for an Agent Evaluation Analyst.Key Responsibilities Take ownership of delivery team projects, ensuring data quality and timely delivery Lead a cross-functional team to keep...Show more A company is looking for an AI Agent Developer.Key Responsibilities Architect and implement AI-powered agents using Large Language Models (LLMs) Connect agents to internal systems and databases,...Show more A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more 
Pilot—CBP Air Interdiction Agent Air and Marine Operations (AMO), a component of U.Customs and Border Protection (CBP), offers skilled Pilots interested in law enforcement an opportunity to work wi...Show more A company is looking for a Multimodal AI Evaluation Analyst.Key Responsibilities Evaluate and score AI-generated outputs across various modalities Assess quality, correctness, coherence, style, ...Show more A company is looking for a Software Engineer II to evaluate AI multimedia generation products for images and videos.Key Responsibilities Set up Halo tasks and run model inference Audit annotatio...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show more A company is looking for a Data Specialist to assist in training AI applications through video and audio annotation and prompt evaluation.
Key Responsibilities Annotate audio events within video c...Show more
AI Solutions Analyst
VirtualVocations • Boca Raton, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted
AI Agent Evaluation Analyst
VirtualVocations • Boca Raton, Florida, United States
Full-time
Last updated: 20 days ago • Promoted
Remote Equity Research Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • Lauderhill, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Remote Financial Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Oakland Park, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
AI Multimedia Evaluator
VirtualVocations • Boca Raton, Florida, United States
Full-time
Last updated: 5 days ago • Promoted
Real Estate Agent
Weichert, Realtors • Miami Beach, Florida, US
Full-time
Last updated: 30+ days ago • Promoted
AI Data Project Manager
VirtualVocations • Fort Lauderdale, Florida, United States
Full-time
Last updated: 3 days ago • Promoted
AI Agent Developer
VirtualVocations • Hialeah, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted
AI Evaluation Analyst
VirtualVocations • Boca Raton, Florida, United States
Full-time
Last updated: 20 days ago • Promoted
Remote Investment Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • North Lauderdale, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Air Interdiction Agent
Customs and Border Protection • Key Biscayne, Florida, US
Full-time
Last updated: 30+ days ago • Promoted
Multimodal AI Evaluation Analyst
VirtualVocations • Boca Raton, Florida, United States
Full-time
Last updated: 2 days ago • Promoted
AI Multimedia Generation Evaluator
VirtualVocations • Fort Lauderdale, Florida, United States
Full-time
Last updated: 3 days ago • Promoted
Remote FP&A Manager – AI Trainer ($50-$60 / hour)
Data Annotation • Coconut Creek, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Remote Senior Financial Analyst ($50-$60 / hour)
Data Annotation • Miami Beach, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Remote Corporate Development Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • Coral Springs, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • North Miami, Florida
Remote
Full-time +1
Last updated: 11 days ago • Promoted
Data Specialist for AI Evaluation
VirtualVocations • Hialeah, Florida, United States
Full-time
Last updated: 30+ days ago • Promoted