Research Engineer - Evaluations

About The Role

Luma is pushing the boundaries of generative AI, building tools that redefine how visual content is created. We're seeking a Research Engineer to design and scale the infrastructure that powers our model evaluation efforts. This role is about building the pipelines, metrics, and automated systems that close the loop between model output, evaluation, and improvement. You'll work across research, engineering, and product teams to ensure our models are measured rigorously, consistently, and in ways that directly inform development.

Responsibilities

• Design and implement scalable pipelines for automated evaluation of generative models, with a focus on visual and multimodal outputs (image, video, text, audio).
• Develop novel metrics and evaluation models that capture qualities like fidelity, coherence, temporal consistency, and alignment with human intent.
• Integrate evaluation signals into training loops (including reinforcement learning and reward modeling) to continuously improve model performance.
• Build infrastructure for large-scale regression testing, benchmarking, and monitoring of multimodal generative models.
• Collaborate with researchers running human studies to translate human evaluation frameworks into automated or semi-automated systems.
• Partner with model researchers to identify failure cases and build targeted evaluation harnesses.
• Maintain dashboards, reporting tools, and alerting systems to surface evaluation results to stakeholders.
• Stay current with emerging evaluation techniques in generative AI, multimodal LLMs, and perceptual quality assessment.

Qualifications

Nice to Have

Seniority level

Employment type

Job function

Industries

#J-18808-Ljbffr