AI Model Evaluation Specialist

About the job AI Model Evaluation Specialist

Key Responsibilities :

• Perform scoring and qualitative evaluations ofLLM-generated responses across multiple use cases.
• Develop and maintain scoring guidelines and rubrics toensure consistency and objectivity.
• Collaborate with data scientists, product managers, andengineering teams to align scoring with project goals.
• Assist in the creation and labeling of high-qualityevaluation datasets for prompt tuning or model fine-tuning.
• Utilize NLP-based metrics and tools (e.g., ROUGE, BLEU,cosine similarity) for automated scoring support.
• Document scoring patterns, common model errors, andimprovement opportunities.
• Contribute to prompt experimentation and help compareeffectiveness of different prompt strategies.

Qualifications :