Remote Expert Prompt Curators for Advanced AI Evaluation Dataset - AI Trainer ($50-$70 per hour)Mercor • Frederick, Maryland, US
Remote Expert Prompt Curators for Advanced AI Evaluation Dataset - AI Trainer ($50-$70 per hour)
Mercor • Frederick, Maryland, US
1 day ago
Job type
- Full-time
- Remote
Job description2\. Key Responsibilities Create original, expert-level prompts that require tool use (e.g., search, browse, or code execution). - Ensure prompts are objective, self-contained, and yield clear, unambiguous answers. - Test prompts against advanced AI models and document failures / successes. - Provide reasoning steps and solutions for each prompt. - Classify prompts into subject domains for dataset organization. - Collaborate with reviewers for expert validation and prompt refinement. ## 3\. Ideal Qualifications Advanced academic or professional expertise in a specialized subject (STEM, law, finance, history, cultural studies, etc.). - Strong ability to design precise, high-difficulty questions requiring deep knowledge and external references. - Experience in academic research, benchmarking, or test question design preferred. - Attention to detail and ability to provide concise reasoning explanations. - Familiarity with AI models and their limitations is a plus. ## 4\. More About the Opportunity Remote and asynchronous — set your own hours. - Expected commitment : ~10–20 hours / week. - Project duration : ~2 months, with possible extensions based on dataset needs. - Opportunity to contribute to high-impact AI safety and evaluation research. ## 5\. Compensation & Contract Terms Competitive hourly compensation based on expertise. - Independent contractor engagement. - Payments for services rendered processed weekly via Stripe Connect. ## 6\. Application Process Submit your resume or CV highlighting your subject matter expertise. - Complete a brief questionnaire about your background and areas of specialization. - Selected applicants may be asked to draft a short test prompt. - You’ll receive follow-up within a few days regarding next steps. ## 7\. About Mercor Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations. - Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey. - Thousands of professionals across domains like law, creatives, engineering, and research have joined Mercor to work on frontier projects shaping the next era of AI.
##
- 1\. Role Overview
Mercor is collaborating with a leading AI research lab to develop a next-generation evaluation dataset for frontier AI models. We are seeking experts with advanced domain knowledge across diverse fields to design extremely challenging prompts that cannot be solved by existing AI systems without internet search or browsing capabilities. The goal is to create a benchmark dataset that pushes the limits of current AI reasoning and retrieval. This is a short-term research engagement with significant impact on AI evaluation. ##
Create a job alert for this search
Curator • Frederick, Maryland, US
Related jobs
At Leidos, you'll contribute to AI solutions that serve critical national and global missions-ranging from defense and intelligence to healthcare, energy, and space exploration.Our work emphasizes ...Show more
Advanced Practice Provider Opportunity.Pennsylvania Cancer Specialists & Research Institute.About the Practice and their Mission : .
Pennsylvania Cancer Specialists & Research Institute ....Show more
Inova Analytics is looking for a dedicated AI Product Manager to join the Team.This hybrid role will be full-time day shift working Monday - Friday.
The AI Product Manager owns the enterprise AI pro...Show more
Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...Show more
Strider Technologies is on a mission to deliver strategic intelligence that enables faster, more confident decision-making for organizations around the world.
As the leading strategic intelligence c...Show more
Leidos Government Health & Safety Solution Group in Windsor Mill, MD supports the End User Centric IT Support (ECIS 2.Contract for Centers for Medicare and Medicaid Services (CMS).Our program deliv...Show more
Halvik is a highly successful WOB business with more than 50 prime contracts and 500+ professionals delivering Digital Services, Advanced Analytics, Artificial Intelligence / Machine Learning, Cyber ...Show more
Director of HR Systems, Data, & AI Strategy .Administrative or Professional Faculty .Administrative or Professional Faculty.
Not eligible for visa sponsorship.Salary commensurate with education and ...Show more
Become A 100% Remote Medical Expert Reviewer with.Join Maximus as an Expert Medical Advisor.As an EMA, you will play a critical role in resolving disputes related to reimbursement, differing opinio...Show more
Project Manager / Scrum Master Ai Programs.Ampcus is seeking an experienced, proactive and organized Project Manager / Scrum Master to lead agile delivery and cross-functional coordination for AI init...Show more Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...Show more 
At Leidos, you'll contribute to AI solutions that serve critical national and global missions-ranging from defense and intelligence to healthcare, energy, and space exploration.Our work emphasizes ...Show more
Task Force Talent is seeking mid to senior-level machine learning (AI / ML) engineers (possibly multiple openings, at least one lead position) for a very well-funded early-stage company working on au...Show more
Citizant is a leading provider of professional IT services to the U.We seek to address some of our country's most pressing challenges in the areas of Agile application development, Enterprise Data ...Show more
NYSE : TGNA) helps people thrive in their local communities by providing the trusted local news and services that matter most.
With 64 television stations in 51 U.TEGNA reaches more than 100 million ...Show more
TechTrend is seeking a talented Senior AI Engineer to join our AI Rapid Innovation team.Using a modern technology stack and agile approach, in this role you will have the opportunity to lead the de...Show more
Our client is an enterprise company developing and providing solutions to its customers both commercial and federal.They are committed to fighting financial crime by detecting and investigating ill...Show more
Plateau GRP is seeking a skilled AI Engineer to support a large-scale data modernization initiative for the Federal Deposit Insurance Corporation (FDIC).
The ideal candidate will apply artificial in...Show more
Senior Agentic AI Test and Evaluation Engineer
Leidos Inc • Reston, VA, United States
Full-time
Last updated: 30+ days ago • Promoted
Advanced Practice Provider - Pennsylvania Cancer Specialists & Research Institute - Gettysburg, PA
OneOncology • Gettysburg, PA, US
Full-time +1
Last updated: 30+ days ago • Promoted
AI Product Manager
Inova Health System • Fairfax, VA, United States
Full-time
Last updated: 18 hours ago • Promoted • New!
AI Trainer -Remote Content QA Reviewer
Outlier • Germantown, MD, United States
Remote
Full-time
Last updated: 2 days ago • Promoted
Senior AI Engineer
Strider Technologies • Tysons, Virginia, United States
Full-time
Last updated: 18 days ago • Promoted
Microsoft Copilot Studio-AI Expert
Leidos Inc • Reston, VA, United States
Full-time
Last updated: 30+ days ago • Promoted
Machine Learning Engineer - Remote
Halvik • Vienna, VA, USA
Remote
Full-time
Quick Apply
Last updated: 30+ days ago
Director of HR Systems, Data, & AI Strategy
George Mason University • Fairfax, VA, United States
Full-time
Last updated: 30+ days ago • Promoted
100% Remote | Reproductive Endocrinology Providers Needed for Medical Reviews
Maximus Federal & VES • McLean, VA, US
Remote
Permanent
Last updated: 30+ days ago • Promoted
Project Manager / Scrum Master AI Programs
Ampcus • Chantilly, VA, US
Full-time
Last updated: 30+ days ago • Promoted
AI Trainer -Remote Writing Evaluator
Outlier • Germantown, MD, United States
Remote
Full-time
Last updated: 2 days ago • Promoted
Senior Agentic AI Test and Evaluation Engineer
Leidos • Reston, VA, United States
Full-time
Last updated: 5 days ago • Promoted
Mid / Senior AI / ML Engineer (U.S. Citizens / LPR Only)
Taskforcetalent • Tysons Corner, VA, United States
Full-time +1
Last updated: 30+ days ago • Promoted
AI Solution Architect - 100% Remote (REF1868Q)
Citizant • Chantilly, VA, United States
Remote
Full-time
Last updated: 30+ days ago • Promoted
AI Engineer (Prompt)
Tegna • Tysons, Virginia, United States
Full-time +1
Last updated: 30+ days ago • Promoted
Senior AI Engineer (Reston)
TechTrend, Inc. • Reston, VA, US
Full-time +1
Last updated: 1 day ago • Promoted
Hybrid Applied AI Engineer- FS Poly- up to $275K
stanleyreid.com • Herndon, VA, United States
Full-time
Last updated: 5 days ago • Promoted
AI Engineer
Plateau Software • Oakton, Virginia, United States
Full-time
Last updated: 8 days ago • Promoted