Expert Prompt Curators for Advanced AI Evaluation DatasetMercor • DeKalb, Illinois, US
No longer accepting applications
Expert Prompt Curators for Advanced AI Evaluation Dataset
Mercor • DeKalb, Illinois, US
1 day ago
Job type
- Full-time
- Remote
Job description2\. Key Responsibilities Create original, expert-level prompts that require tool use (e.g., search, browse, or code execution). - Ensure prompts are objective, self-contained, and yield clear, unambiguous answers. - Test prompts against advanced AI models and document failures / successes. - Provide reasoning steps and solutions for each prompt. - Classify prompts into subject domains for dataset organization. - Collaborate with reviewers for expert validation and prompt refinement. ## 3\. Ideal Qualifications Advanced academic or professional expertise in a specialized subject (STEM, law, finance, history, cultural studies, etc.). - Strong ability to design precise, high-difficulty questions requiring deep knowledge and external references. - Experience in academic research, benchmarking, or test question design preferred. - Attention to detail and ability to provide concise reasoning explanations. - Familiarity with AI models and their limitations is a plus. ## 4\. More About the Opportunity Remote and asynchronous — set your own hours. - Expected commitment : ~10–20 hours / week. - Project duration : ~2 months, with possible extensions based on dataset needs. - Opportunity to contribute to high-impact AI safety and evaluation research. ## 5\. Compensation & Contract Terms Competitive hourly compensation based on expertise. - Independent contractor engagement. - Payments for services rendered processed weekly via Stripe Connect. ## 6\. Application Process Submit your resume or CV highlighting your subject matter expertise. - Complete a brief questionnaire about your background and areas of specialization. - Selected applicants may be asked to draft a short test prompt. - You’ll receive follow-up within a few days regarding next steps. ## 7\. About Mercor Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations. - Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey. - Thousands of professionals across domains like law, creatives, engineering, and research have joined Mercor to work on frontier projects shaping the next era of AI.
##
- 1\. Role Overview
Mercor is collaborating with a leading AI research lab to develop a next-generation evaluation dataset for frontier AI models. We are seeking experts with advanced domain knowledge across diverse fields to design extremely challenging prompts that cannot be solved by existing AI systems without internet search or browsing capabilities. The goal is to create a benchmark dataset that pushes the limits of current AI reasoning and retrieval. This is a short-term research engagement with significant impact on AI evaluation. ##
Create a job alert for this search
Curator • DeKalb, Illinois, US
Related jobs
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.
We guarantee 15-25 hours per week with an hourly pay of bet...Show more
Our client is a leading force in advancing safer, smarter AI technology.Their work has been featured in.They’ve built a global community of expert contributors and have already paid out more ...Show more
Senior Project Engineer / Environmental — Chicago Suburbs Region.Lead environmental studies and permitting projects.Manage technical staff and project budgets.
Prepare reports and client prese...Show more
Dekalb, Illinois (One hour from Chicago).Healthcare United is seeking a.Board Eligible / Board Certified Endocrinologist.
The selected candidate will provide dedicated support to an existing.Work alon...Show more
This position has opportunity to develop into many other career opportunities within the company.This position performs various clerical tasks that require great attention to detail.The successful ...Show more
Join a global community of talented professionals to shape the future of AI.Earn up to $15 USD / hr and additional rewards based on quality of submission.
Outlier is committed to improving the intelli...Show more
Hematology-Oncology physician job in Illinois : Some locum assignments can be as short as a day, others, years.Some are far from home, others are local.
Whatever it is you're looking for, we offer t...Show more
Are you looking to earn extra income from the comfort of your home? We're seeking motivated individuals to explore and test a variety of remote side hustle opportunities featured on FinanceBuzz.Thi...Show more Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.
We guarantee 15-25 hours per week with an hourly pay of bet...Show more 
Join a team that truly supports you—personally, professionally, and clinically.Sandwich Veterinary Hospital, a thriving 3-doctor practice in beautiful northern Illinois, is excited to welcome a ful...Show more
Oral & Maxillofacial Surgeons for Northern Illinois, the practice of Dr.David Junck DDS, is looking for an associate to help them continue to grow their full-scope practice in Sycamore, IL.The part...Show more)
WHEN YOU WORK FOR US, WE WORK FOR YOU.With Titan Medical, you gain access to thousands of travel nursing and allied health jobs across the country.
You also get unmatched service.From the moment you...Show more
Nationwide Therapy Group is seeking a travel nurse RN Long Term Care for a travel nursing job in Harvard, Illinois.Job Description & Requirements.
Mercyhealth Hospital and Medical Center-Harvard.Reg...Show more
Rang Healthcare is seeking a travel nurse RN Skilled Nursing for a travel nursing job in Harvard, Illinois.Job Description & Requirements.
We are seeking an experienced RN for our Skilled Nursing un...Show more
Join an established Primary Care practice.Perform patient assessment and assign diagnosis based on recorded assessment data and current scientific and current research.
Establish plan of care and im...Show more Data Analysis - AI Training Role.Our client is a leading force in advancing safer, smarter AI technology.Their work has been featured in.
They have built a global community of expert contributors an...Show more 
Cytotechnologist ( ASCP ) ( CT ).Cytotechnologist to join our team and play a crucial role in diagnosing diseases, with a particular focus on cancer detection.
In this position, you will examine and...Show more Nationwide Therapy Group is seeking a travel nurse RN Long Term Care for a travel nursing job in Harvard, Illinois.Job Description & Requirements.
Day Shift – 7 : 00 AM to 7 : 30 PM (12.Active Illinois ...Show more
Test Products from Home – $25-$45 / hr + Freebies
OCPA • Plano, Illinois, us
Part-time +1
Last updated: 30+ days ago • Promoted
Engineer
Quality Talent Group • DeKalb, Illinois, United States
Full-time
Quick Apply
Last updated: 7 days ago
Senior Project Engineer / Environmental - Chicago Suburbs Region
Gpac • Plano, Illinois, United States
Full-time
Quick Apply
Last updated: 2 days ago
Endocrinologist Needed Near Dekalb, IL (1 hour West Chicago)
Healthcare United • DeKalb, IL, United States
Full-time
Last updated: 30+ days ago • Promoted
Title Search Assistant
Fox Title Company • DeKalb, Illinois, United States
Full-time
Quick Apply
Last updated: 30+ days ago
generalists
Scale AI • DeKalb, Illinois, United States
Full-time
Last updated: 9 days ago • Promoted
Hematology-Oncology
CompHealth • DeKalb, Illinois, US
Full-time
Last updated: 30+ days ago • Promoted
Remote Side Hustle Evaluator - Flexible Online Gig Work
Finance Buzz • Sandwich, Illinois, US
Remote
Temporary
Last updated: 30+ days ago • Promoted
Remote Product Tester – $45 / hr + Free Products – Start Now!
OCPA • Plano, Illinois, us
Remote
Part-time +1
Last updated: 30+ days ago • Promoted
Veterinarian
National Veterinary Associates • SANDWICH, Illinois, United States
Full-time +1
Last updated: 30+ days ago • Promoted
Private Practice in Chicago Suburb Expanding and Seeking Associate!
Oral & Maxillofacial Surgeons for Northern Illinois • Sycamore, IL, United States
Full-time
Last updated: 8 days ago • Promoted
Travel - Cath Lab Tech
Titan Medical (Nebraska) • Sycamore, IL, United States
Full-time
Last updated: 5 hours ago • Promoted • New!
Travel Nurse RN - Long Term Care - $1,858 per week
Nationwide Therapy Group • Harvard, IL, United States
Full-time
Last updated: 15 days ago • Promoted
Travel Nurse RN - Skilled Nursing - $1,948 per week
Rang Healthcare • Harvard, IL, United States
Permanent
Last updated: 30+ days ago • Promoted
Advanced Practice Provider Primary Care
OSF HealthCare • Poplar Grove, IL, United States
Full-time
Last updated: 30+ days ago • Promoted
Data Analysis
Quality Talent Group • DeKalb, Illinois, United States
Full-time
Quick Apply
Last updated: 7 days ago
ASCP - Cytotechnologist
MLee Healthcare Staffing and Recruiting, Inc • Kingston, IL, United States
Full-time +1
Last updated: 4 days ago • Promoted
Travel Nurse RN - Long Term Care - $1,818 per week
Nationwide Therapy Group • Harvard, IL, United States
Full-time
Last updated: 15 days ago • Promoted