Expert Prompt Curators for Advanced AI Evaluation DatasetMercor • Redmond, Washington, US
No longer accepting applications
Expert Prompt Curators for Advanced AI Evaluation Dataset
Mercor • Redmond, Washington, US
1 day ago
Job type
- Full-time
- Remote
Job description2\. Key Responsibilities Create original, expert-level prompts that require tool use (e.g., search, browse, or code execution). - Ensure prompts are objective, self-contained, and yield clear, unambiguous answers. - Test prompts against advanced AI models and document failures / successes. - Provide reasoning steps and solutions for each prompt. - Classify prompts into subject domains for dataset organization. - Collaborate with reviewers for expert validation and prompt refinement. ## 3\. Ideal Qualifications Advanced academic or professional expertise in a specialized subject (STEM, law, finance, history, cultural studies, etc.). - Strong ability to design precise, high-difficulty questions requiring deep knowledge and external references. - Experience in academic research, benchmarking, or test question design preferred. - Attention to detail and ability to provide concise reasoning explanations. - Familiarity with AI models and their limitations is a plus. ## 4\. More About the Opportunity Remote and asynchronous — set your own hours. - Expected commitment : ~10–20 hours / week. - Project duration : ~2 months, with possible extensions based on dataset needs. - Opportunity to contribute to high-impact AI safety and evaluation research. ## 5\. Compensation & Contract Terms Competitive hourly compensation based on expertise. - Independent contractor engagement. - Payments for services rendered processed weekly via Stripe Connect. ## 6\. Application Process Submit your resume or CV highlighting your subject matter expertise. - Complete a brief questionnaire about your background and areas of specialization. - Selected applicants may be asked to draft a short test prompt. - You’ll receive follow-up within a few days regarding next steps. ## 7\. About Mercor Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations. - Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey. - Thousands of professionals across domains like law, creatives, engineering, and research have joined Mercor to work on frontier projects shaping the next era of AI.
##
- 1\. Role Overview
Mercor is collaborating with a leading AI research lab to develop a next-generation evaluation dataset for frontier AI models. We are seeking experts with advanced domain knowledge across diverse fields to design extremely challenging prompts that cannot be solved by existing AI systems without internet search or browsing capabilities. The goal is to create a benchmark dataset that pushes the limits of current AI reasoning and retrieval. This is a short-term research engagement with significant impact on AI evaluation. ##
Create a job alert for this search
Curator • Redmond, Washington, US
Related jobs
We are seeking a Research Engineer for our team that researches new wearable AI devices and interaction models.This role combines development of interactive AI experiences and research of human beh...Show more
SystImmune is a leading and well-funded clinical-stage biopharmaceutical company located in Redmond, WA and Princeton, NJ.
It specializes in developing innovative cancer treatments using its establi...Show more
Need to have a strong background in Azure development, with expertise in C# / Python, Azure Open AI, Agentic workflow, AutoGen, and Azure.
The candidate should also possess a deep understanding of s...Show more
Human Subjects Research Assistant.US Citizen, GC Holders or Authorized to Work in the U.Our goal is to explore, innovate and design novel interfaces and hardware subsystems for the next generation ...Show more
HCLTech is looking for a highly talented and self- motivated [Senior Adobe Creative Automation & Generative AI Developer] to join it in advancing the technological world through innovation and crea...Show more
We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show more
SystImmune is a leading and well-funded clinical-stage biopharmaceutical company located in Redmond, WA and Princeton, NJ.
It specializes in developing innovative cancer treatments using its establi...Show more
We are looking for a highly skilled.C# development, Azure AI Foundry, and multi-agent orchestration.If you have hands-on experience building scalable AI systems and working within the Microsoft eco...Show more
A leading mid-sized biotech is seeking a highly skilled Machine Learning professional with experience in data mining to analyze large datasets from public domain sources and internal data.The ideal...Show more
Amazon Leo is an initiative to launch a constellation of Low Earth Orbit satellites that will provide low-latency, high-speed broadband connectivity to unserved and underserved communities around t...Show more
At Centific, people are at the center of our culture.We constantly seek out opportunities for people to enhance their skills,.
We believe that competition can bring out the very best in people.AI ch...Show more
I hope this message finds you well.I’m reaching out to share an exciting opportunity that I believe aligns with your experience and expertise in the field for my client.
If you are interested, I’d l...Show more SystImmune is a leading and well-funded clinical-stage biopharmaceutical company located in Redmond, WA and Princeton, NJ.
It specializes in developing innovative cancer treatments using its establi...Show more 
We’re seeking technical professionals passionate about building intelligent systems and automation solutions that drive real-world impact.
This role sits at the cutting edge of AI innovation—combini...Show more
We are seeking participants for Project A, an onsite paid research study focused оn improving the usability and comfort оf next-generation smart glasses.
Your insights will directly influence the de...Show more
Location : Onsite in Redmond, WA.C++ / C# in a game engine (Unity, Unreal), Unity is preferred.Python for data analysis (numpy, scipy).
Familiarity with user research, behavioral experiments.Familiarit...Show more
Turn your passion for luxury into a career opportunity.Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles.
Join CXG, the global leader in custo...Show more
At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio.We are currently seeking a Automation and.
If you're looking for an exciting opportunity to grow in a in...Show more
Research Engineer (Redmond)
Intelliswift - An LTTS Company • Redmond, WA, US
Part-time
Last updated: 4 hours ago • Promoted • New!
Machine Learning Frontier Scientist - AI Drug Discovery
SystImmune Inc. • Redmond, WA, United States
Full-time
Last updated: 30+ days ago • Promoted
Azure AI Developer
Omni Inclusive • Redmond, WA, United States
Full-time
Last updated: 30+ days ago • Promoted
Human Subjects Research Assistant
INSPYR Solutions • Redmond, WA, US
Full-time
Last updated: 30+ days ago • Promoted
Senior Adobe Creative Automation & Generative AI Developer (Redmond)
HCLTech • Redmond, WA, US
Full-time +1
Last updated: 4 hours ago • Promoted • New!
Market Research Contributor for Fortune 500 Companies (Hiring Immediately)
Earn Haus • Enumclaw, Washington, US
Full-time +2
Last updated: 30+ days ago • Promoted
Scientist / Research Associate – AI Protein Design
Systimmune • Redmond, WA, US
Full-time
Quick Apply
Last updated: 30+ days ago
AI Engineer
Quadrant Technologies • Redmond, WA, United States
Full-time
Last updated: 2 hours ago • Promoted • New!
Machine Learning / Frontier Scientist
BioTalent • Redmond, WA, United States
Full-time
Last updated: 30+ days ago • Promoted
Sr Product Design Eng - CT, Amazon Leo
Amazon Kuiper Manufacturing Enterprises LLC • Redmond, WA, US
Permanent
Last updated: 1 day ago • Promoted
Data Evaluator
Centific Global Solutions, Inc. • Redmond, Washington
Full-time
Last updated: 10 days ago
GENAI / MACHINE LEARNING AI / DATA Scientist
Hudson Manpower • Issaquah, WA, US
Full-time
Last updated: 30+ days ago
Senior / Principal Scientist – AI / ML Protein & Antibody Drug Discovery
Systimmune • Redmond, WA, US
Full-time
Quick Apply
Last updated: 30+ days ago
Technical AI Engineer / Developer
AG Consulting Partners, Inc. • Redmond, WA, US
Full-time
Quick Apply
Last updated: 30+ days ago
Project A – Paid Onsite Research Study (Redmond, WA)
OneForma • Redmond, WA, United States
Full-time
Last updated: 2 hours ago • Promoted • New!
Research Engineer (Redmond)
BeaconFire Inc. • Redmond, WA, US
Part-time
Last updated: 1 hour ago • Promoted • New!
Become a Luxury Brand Evaluator in Bellevue, WA- Apply Now
CXG • Redmond, WA, US
Full-time
Quick Apply
Last updated: 30+ days ago
Automation and AI Specialist
TechBiz Global GmbH • Redmond, WA, US
Full-time
Last updated: 2 days ago