LLM Quality Analyst

A company is looking for a LLM Quality Analyst to support the quality and reliability of AI outputs.

Key Responsibilities

Manage LLM feedback intake and triage, identifying patterns in quality issues

Develop and maintain evaluation test suites, including generating ground truth datasets

Support evaluator and judge development by drafting prompts and scoring rubrics

Required Qualifications

1-3 years of professional experience or relevant internship experience

Education background in education or related field

Experience with LLMs or AI products preferred

Passionate about solving education problems with technology

Excited to explore different areas of tech, such as product and data analysis