Audio Generalist Evaluator Expert

Mercor is seeking detail-oriented writing experts to contribute to a high-impact audio AI research project with a leading lab. Freelancers will author prompt–golden answer pairs that train and evaluate advanced language models. This is a short-term, flexible opportunity for professionals with strong academic backgrounds and a knack for instructional clarity. Ideal for those who enjoy distilling complex concepts into well-crafted text.

• ###
• Job Details :
• Design and Optimize Prompts
• : Create detailed audio prompts with multiple constraints and instructions. -
• Define and Document Evaluation Standards
• : Establish high-level expectations for correct responses in general audio consumer contexts, and develop comprehensive rubric. -
• Conduct Model Testing and Grading
• : Run prompts through models and assess preliminary outputs against expectations. -
• Support Benchmarking and Quality Assurance
• : Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor, maintaining consistency and reliability before integration into official benchmarks. ###
• Minimum Qualifications :
• BS or BA from a reputable institution completed or in progress - Strong writing and critical thinking skills. - Ability to work independently and meet deadlines. - Significant familiarity with ChatGPT or similar tools for personal decision-making or hobbies / general interests. ###
• Preferred Qualifications :
• 2+ years of experience in teaching or research. ###
• Application & Onboarding Process :
• Complete an AI-led interview, this should take around 15 minutes. - Complete a 45-minute written assessment that will guide you through writing rubrics. - If selected, you will be invited to work on the project. ###
• More Details About This Role :
• This is a
• remote and asynchronous
• role — work on your own schedule. - Expect to contribute at least
• 20 hours per week
• . - Expect a commitment of around 1 month. - You’ll be working in a structured project environment with clear goals and tools.
• ###
• About
• Mercor
• ](https : / / mercor.com / )
• Our team is based in San Francisco, CA - We [ in recruiting experts for top AI labs - Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey