Talent.com
Senior Research Scientist, Model Evaluation
Senior Research Scientist, Model EvaluationCohere • San Francisco, CA, United States
Senior Research Scientist, Model Evaluation

Senior Research Scientist, Model Evaluation

Cohere • San Francisco, CA, United States
2 days ago
Job type
  • Full-time
Job description

Senior Research Scientist, Model Evaluation

Join to apply for the Senior Research Scientist, Model Evaluation role at Cohere .

Why this role?

Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating next?generation evaluation methods and infrastructure to measure LLM progress.

As a Senior Research Scientist, Model Evaluation, You Will

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
  • Work on highly cross?functional teams to translate model feedback into trustworthy, repeatable evaluations.
  • Conduct research to advance the state?of?the?art in LLM evaluation methods, including training LLM judges; refining LLM?based data synthesis pipelines; and improving evaluation efficiency.
  • Build scalable and reusable tools for digging into model performance.

You May Be a Good Fit If

  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
  • You have strong software engineering skills.
  • We value and celebrate diversity

    We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.

    Full?Time Employees At Cohere Enjoy These Perks

  • ???? An open and inclusive culture and work environment
  • ???????? Work closely with a team on the cutting edge of AI research
  • ???? Weekly lunch stipend, in?office lunches & snacks
  • ???? Full health and dental benefits, including a separate budget to take care of your mental health
  • ???? 100% Parental Leave top?up for up to 6 months
  • ???? Personal enrichment benefits towards arts and culture, fitness and well?being, quality time, and workspace improvement
  • ???? Remote?flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co?working stipend
  • ?? 6 weeks of vacation (30 working days!)
  • Seniority level

    Mid?Senior level

    Employment type

    Full?time

    Job function

    Other

    Industries

    Software Development

    #J-18808-Ljbffr

    Create a job alert for this search

    Senior Research Scientist Model Evaluation • San Francisco, CA, United States

    Similar jobs
    Senior Applied Research Scientist - Foundation Models

    Senior Applied Research Scientist - Foundation Models

    Ambient AI • Redwood City, CA, United States
    Full-time
    Build a safer world with us, one incident at a time.AI-powered physical security platform helping the world's leading enterprises reduce risk, improve operational efficiency, and gain critical insi...Show more
    Last updated: 2 days ago • Promoted
    Research Scientist : AI Evaluation & Alignment

    Research Scientist : AI Evaluation & Alignment

    Patronus AI • San Francisco, CA, United States
    Full-time
    A leading AI company is seeking a Research Scientist to develop cutting-edge AI evaluation systems and conduct transformative research in language models. Candidates should have a background in empi...Show more
    Last updated: 26 days ago • Promoted
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Together AI • San Francisco, CA, United States
    Full-time
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis.Together AI is building the fastest, most capable open‑source‑aligned LLMs and inference stack in the world.As part of the Turbo org...Show more
    Last updated: 30+ days ago • Promoted
    Assistant / Associate Project Scientist Neuroimaging Biomarkers of Alzheimers disease

    Assistant / Associate Project Scientist Neuroimaging Biomarkers of Alzheimers disease

    InsideHigherEd • Berkeley, California, United States
    Full-time
    Assistant / Associate Project Scientist Neuroimaging Biomarkers of Alzheimers disease Helen Wills Neuroscience Institute. The UC academic salary scales set the minimum pay determined by rank and step...Show more
    Last updated: 30+ days ago • Promoted
    Scientist / Senior Scientist, Display Technologies

    Scientist / Senior Scientist, Display Technologies

    BigHat Biosciences • San Mateo County, California, US
    Full-time
    Job Description Job Description Description The Role : BigHat Biosciences is seeking a self-motivated Scientist or Senior Scientist to join the Display Technologies team. This individual will expand...Show more
    Last updated: 7 days ago • Promoted
    Senior Research Scientist, Foundation Model (LLM / VLM)

    Senior Research Scientist, Foundation Model (LLM / VLM)

    Waymo • San Francisco, CA, United States
    Full-time
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...Show more
    Last updated: 2 days ago • Promoted
    Research Scientist, World Models

    Research Scientist, World Models

    Waabi • San Francisco, California, US
    Full-time
    Job Description Job Description Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI.With a world-class team, we're unlocking the next era of autonomous transportation with...Show more
    Last updated: 7 days ago • Promoted
    Senior AI Research Scientist

    Senior AI Research Scientist

    Workato • San Francisco, California, US
    Full-time
    Job Description Job Description About Workato Workato transforms technology complexity into business opportunity.As the leader in enterprise orchestration, Workato helps businesses globally strea...Show more
    Last updated: 7 days ago • Promoted
    Senior Scientist I / II - Analytical Development (CMC)

    Senior Scientist I / II - Analytical Development (CMC)

    Mammoth Biosciences • Brisbane, California, US
    Full-time
    Job Description Job Description OPPORTUNITY The Senior Scientist will be responsible for managing analytical development, guiding external partners (e. CTLs, CDMOs), and standing up key methods i...Show more
    Last updated: 7 days ago • Promoted
    Senior Applied Research Scientist

    Senior Applied Research Scientist

    Tensorlake Inc. • San Francisco, CA, United States
    Full-time
    Tensorlake is a serverless platform for agentic applications and Document Ingestion.It provides foundational infrastructure such as serverless compute, code sandboxes, durable execution and unstruc...Show more
    Last updated: 2 days ago • Promoted
    Senior Applied Research Scientist - Foundation Models

    Senior Applied Research Scientist - Foundation Models

    Ambient • Redwood City, CA, United States
    Full-time
    Senior Applied Research Scientist.AI-powered physical security platform helping the world's leading enterprises reduce risk, improve operational efficiency, and gain critical insights.Fortune 500 o...Show more
    Last updated: 11 hours ago • Promoted • New!
    Associate Scientist, Advanced Characterization

    Associate Scientist, Advanced Characterization

    OmniAb • Emeryville, California, US
    Full-time
    Job Description Job Description About OmniAb OmniAb provides our pharmaceutical industry partners access to the most diverse antibody repertoires and cutting-edge screening technologies to enable...Show more
    Last updated: 7 days ago • Promoted
    Associate Scientist / Scientist - Analytical Development

    Associate Scientist / Scientist - Analytical Development

    ZYLIDAC BIO LLC • Emeryville, California, US
    Full-time
    Job Description Job Description Description : • Perform routine and novel analytical testing to support biological therapeutic products from early-stage development through late- stage / commerciali...Show more
    Last updated: 7 days ago • Promoted
    Research Scientist Manager - ML Modeling & Applied Research

    Research Scientist Manager - ML Modeling & Applied Research

    Meta Platforms • Burlingame, CA, United States
    Full-time
    Research Scientist Manager - ML Modeling & Applied Research.Reality Labs at Meta is seeking Research Scientist Manager with experience in machine learning research to advance our pioneering work in...Show more
    Last updated: 2 days ago • Promoted
    Senior Reward Modeling Scientist - RLHF & LLM Evaluation

    Senior Reward Modeling Scientist - RLHF & LLM Evaluation

    Anthropic • San Francisco, CA, United States
    Full-time
    A leading AI research organization in San Francisco seeks a Senior Research Scientist to enhance reward modeling methods. This position focuses on innovating architectures and methodologies in RLHF ...Show more
    Last updated: 30+ days ago • Promoted
    Senior Machine Learning Engineer - Model Evaluations, Public Sector

    Senior Machine Learning Engineer - Model Evaluations, Public Sector

    Scale AI • San Francisco, CA, United States
    Full-time
    Machine Learning Engineer - Model Evaluations, Public Sector.Louis, MO; New York, NY; Washington, DC.The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, a...Show more
    Last updated: 11 days ago • Promoted
    Senior Research Scientist (LLMs)

    Senior Research Scientist (LLMs)

    Aldea • San Francisco, CA, United States
    Full-time
    Aldea is a multi-modal foundational AI company reimagining the scaling laws of intelligence.We believe today's architectures create unnecessary bottlenecks for the evolution of software.Our mission...Show more
    Last updated: 2 days ago • Promoted
    Senior Machine Learning Scientist

    Senior Machine Learning Scientist

    Tahoe Therapeutics • South San Francisco, California, US
    Full-time
    Job Description Job Description About Tahoe Therapeutics Tahoe Therapeutics is a biotechnology company pioneering a fundamentally new approach to drug discovery, one that begins with the biology ...Show more
    Last updated: 7 days ago • Promoted