No longer accepting applications
AI Agent Evaluation Analyst
VirtualVocationsClearwater, Florida, United States2 days ago
Job type
- Full-time
Job description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats like JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
Create a job alert for this search
Ai Analyst • Clearwater, Florida, United States
Related jobs Key Responsibilities Design, develop, and deploy production-ready AI agents that support critical workflows across the enterprise Rapidly prototype, test, and refine agents to ensure high perfor...Show moreLast updated: 1 day ago A company is looking for an Analytics Engineer to support their operations team by leveraging data to understand user needs and improve customer experiences.
Key Responsibilities Refine data, buil...Show moreLast updated: 30+ days ago A company is looking for an EDI Analyst responsible for internal EDI processes and implementation.Key Responsibilities Maintain knowledge of employee benefit systems and processes related to EDI ...Show moreLast updated: 15 hours ago A company is looking for an AI Engineer, Principal.Key Responsibilities Collaborate with data scientists and ML engineers to containerize, deploy, and monitor AI / ML models Design and manage clou...Show moreLast updated: 30+ days ago A company is looking for a Generative AI Specialist - Humanities (English and Slovak).Key Responsibilities Evaluate and rate the performance of AI models based on output Annotate and classify co...Show moreLast updated: 30+ days ago A company is looking for a Junior Machine Learning Engineer to support the design, development, and deployment of machine learning models.
Key Responsibilities Support the end-to-end lifecycle of ...Show moreLast updated: 30+ days ago A company is looking for a Generative Machine Learning Engineer.Key Responsibilities Develop and implement Deep Learning Pipelines using cutting-edge models Sustain the infrastructure for Deep L...Show moreLast updated: 30+ days ago A company is looking for an AI Developer to advance the development of intelligent applications remotely.Key Responsibilities : Design and develop AI applications with OpenAI GPT models, ensuring ...Show moreLast updated: 30+ days ago A company is looking for an AI / ML Engineer.Key Responsibilities Develop software in Python, C#, and C++ for LLM training, testing, and inference modules Lead the creation and implementation of A...Show moreLast updated: 30+ days ago A company is looking for a Senior Consultant AI.Key Responsibilities Utilize Azure AI Services to build and optimize OpenAI solutions Develop and manage data ingestion and transformation pipelin...Show moreLast updated: 30+ days ago A company is looking for a BI Developer to transform data into actionable insights supporting strategic decision-making.Key Responsibilities Design, develop, deploy, and document interactive dash...Show moreLast updated: 30+ days ago A company is looking for an AI Solutions Specialist to drive business innovation through artificial intelligence.Key Responsibilities Analyze complex datasets to uncover trends and actionable ins...Show moreLast updated: 1 day ago 
All of the programs we support require.All work must be conducted within the continental U.Distributed Data Systems, Platforms at Scale, and Complex Application Development, with headquarters in Mc...Show moreLast updated: 29 days ago A company is looking for a Deep Learning Software Engineer, Inference and Model Optimization - New College Grad 2025.Key Responsibilities Train, develop, and deploy generative AI models using the...Show moreLast updated: 1 day ago A company is looking for an AI Implementation Specialist for Automated Portfolio Analysis and Reporting.Key Responsibilities Develop a GUI-based prototype for AI-powered document analysis and vis...Show moreLast updated: 30+ days ago A company is looking for a Senior Business Intelligence Developer - Product & Platform Innovation.Key Responsibilities Collaborate with cross-functional teams to drive data enablement and support...Show moreLast updated: 30+ days ago A company is looking for a Generative AI Scientist.Key Responsibilities Deliver solutions to identify payment integrity issues and improve healthcare quality Develop exploratory data analysis ap...Show moreLast updated: 1 day ago Machine Learning Engineer to contribute to machine learning solutions for network security challenges.Key Responsibilities Contribute to all stages of AI / ML projects, from exploration to producti...Show moreLast updated: 30+ days ago A company is looking for a Machine Learning Engineer - Inventory Demand Forecasting (Retail / CPG).Key Responsibilities Build ML-based forecasting and inventory optimization systems for retail / eC...Show moreLast updated: 30+ days ago A company is looking for an AI Software Engineer to join their Operations Engineering team.Key Responsibilities Design, develop, and deploy AI-driven features and intelligent agents for real-worl...Show moreLast updated: 30+ days ago
- Promoted
Senior AI Agent Developer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
Analytics Engineer
VirtualVocationsClearwater, Florida, United StatesFull-time
- Promoted
- New!
EDI Analyst
VirtualVocationsTampa, Florida, United StatesFull-time
- Promoted
AI Engineer
VirtualVocationsClearwater, Florida, United StatesFull-time
- Promoted
Generative AI Specialist
VirtualVocationsLargo, Florida, United StatesFull-time
- Promoted
Junior Machine Learning Engineer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
Generative Machine Learning Engineer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
AI Developer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
AI / ML Engineer
VirtualVocationsLargo, Florida, United StatesFull-time
- Promoted
Senior AI Consultant
VirtualVocationsTampa, Florida, United StatesFull-time
- Promoted
Business Intelligence Developer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
AI Solutions Specialist
VirtualVocationsClearwater, Florida, United StatesFull-time
- Promoted
AI / ML Engineer
RaftTampa, FL, United StatesFull-time
- Promoted
Deep Learning Software Engineer
VirtualVocationsClearwater, Florida, United StatesFull-time
- Promoted
AI Implementation Specialist
VirtualVocationsClearwater, Florida, United StatesFull-time
- Promoted
Senior Business Intelligence Developer
VirtualVocationsTampa, Florida, United StatesFull-time
- Promoted
Generative AI Scientist
VirtualVocationsTampa, Florida, United StatesFull-time
- Promoted
Senior Machine Learning Engineer
VirtualVocationsTampa, Florida, United StatesFull-time
- Promoted
Machine Learning Engineer
VirtualVocationsSaint Petersburg, Florida, United StatesFull-time
- Promoted
AI Software Engineer
VirtualVocationsLargo, Florida, United StatesFull-time