Applied Scientist (Evaluation & Safety)

Job Description

Job Description

About Amigo

Amigo builds trust and safety infrastructure for AI in mission-critical environments.

We partner with organizations in healthcare and other regulated sectors to deploy AI systems that operate reliably when the stakes are highest. Our infrastructure enables verification, monitoring, and real-time oversight—ensuring AI serves people safely at scale.

We've raised $6.5M from General Catalyst and GSV Ventures. Our team combines expertise in distributed systems, quantitative research, clinical operations, and regulatory environments to build AI that organizations can trust.

About this role

As an Applied Scientist - Evaluation & Safety at Amigo, you'll build the evaluation frameworks that ensure our systems are safe and effective in healthcare environments. You'll design adversarial testing systems, create safety evaluation protocols, and build frameworks that catch potential failures before they reach patients. Your work is critical to maintaining the reliability and safety standards that healthcare organizations require.

What you'll do

Design evaluation frameworks for healthcare systems

Build adversarial testing systems that identify potential failure modes before deployment

Create safety evaluation protocols that ensure systems meet healthcare standards

Design test suites that cover edge cases, rare diseases, and complex medical scenarios

Build automated evaluation pipelines that continuously assess system performance

Collaborate with healthcare professionals to develop clinically relevant evaluation metrics

What we're looking for

Strong background in system safety and adversarial testing

Experience with bias detection and fairness testing

Knowledge of healthcare safety standards

Understanding of statistical evaluation methods

Track record of building evaluation systems for production software

Nice to have

Healthcare or regulated industry experience

Background in medical device testing or clinical trial design

Experience with formal verification or safety-critical systems

Knowledge of healthcare compliance and audit requirements

Benefits

Health & Wellness

Comprehensive health, dental, and vision insurance

Mental health support and wellness coaching

Flexible wellness stipend for fitness, therapy, or personal growth

Daily catered lunch and dinner

Growth & Development

Annual learning budget for courses, books, or conferences

Conference attendance budget for professional development

Development setup of your choice

Academic collaboration opportunities

Compensation Range : $150K - $300K