Generalist - Labeling

1. Role Overview

Mercor is collaborating with a leading AI lab on a short-term project focused on improving preference ranking models for conversational AI systems. We’re seeking detail-oriented generalists—ideally with prior experience in data labeling or content evaluation—to assess and rank model outputs across a variety of domains. This opportunity is well-suited for professionals comfortable with nuanced judgment tasks and working independently in a remote setup.

2. Key Responsibilities

Evaluate and compare AI-generated responses based on quality, coherence, and helpfulness

Assign preference rankings to pairs or sets of model outputs

Follow detailed labeling guidelines and adjust based on evolving criteria

Provide brief written explanations for ranking decisions when required

Flag edge cases or inconsistencies in task design or model output

3. Ideal Qualifications

Prior experience in data labeling, content moderation, or preference ranking tasks

Excellent critical thinking and reading comprehension skills

Comfort working with evolving guidelines and ambiguity

Strong attention to detail and consistency across repetitive tasks

Availability for regular part-time work on a weekly basis

4. More About the Opportunity

Remote and asynchronous — set your own hours

Expected commitment : 10–20 hours / week

Flexible workload depending on your availability and performance

5. Compensation & Contract Terms

$25–35 / hour depending on experience and location

Payments issued weekly via Stripe Connect

This is a freelance engagement; you’ll be classified as an independent contractor

6. Application Process

Submit your resume to get started

Complete a short form to highlight your relevant experience

You may be asked to complete a brief assessment to evaluate task fit

Expect a response within 3–5 business days

7. About Mercor

Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations

Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey

Thousands of professionals across law, engineering, research, and creative fields collaborate on contract-based projects shaping the future of AI