Talent.com
Staff AI Research Scientist - Data Quality, Handshake AI

Staff AI Research Scientist - Data Quality, Handshake AI

HandshakeSan Francisco, CA, United States
1 day ago
Job type
  • Full-time
Job description

About Handshake AI

Handshake is building the career network for the AI economy. Our three-sided marketplace connects 18 million students and alumni, 1,500+ academic institutions across the U.S. and Europe, and 1 million employers to power how the next generation explores careers, builds skills, and gets hired.

Handshake AI is a human data labeling business that leverages the scale of the largest early career network.We work directly with the world's leading AI research labs to build a new generation of human data products. From PhDs in physics to undergrads fluent in LLMs, Handshake AI is the trusted partner for domain-specific data and evaluation at scale.

This is a unique opportunity to join a fast-growing team shaping the future of AI through better data, better tools, and better systems-for experts, by experts.

Now's a great time to join Handshake. Here's why :

  • Leading the AI Career Revolution : Be part of the team redefining work in the AI economy for millions worldwide.
  • Proven Market Demand : Deep employer partnerships across Fortune 500s and the world's leading AI research labs.
  • World-Class Team : Leadership from Scale AI, Meta, xAI, Notion, Coinbase, and Palantir, just to name a few.
  • Capitalized & Scaling : $3.5B valuation from top investors including Kleiner Perkins, True Ventures, Notable Capital, and more.

About the Role

As a Staff Research Scientist, you will play a pivotal role in shaping the future of large language model (LLM) alignment by leading research and development at the intersection of data quality and post-training techniques such as RLHF, preference optimization, and reward modeling.

You will operate at the forefront of model alignment, with a focus on ensuring the integrity, reliability, and strategic use of supervision data that drives post-training performance. You'll set research direction, influence cross-functional data standards, and lead the development of scalable systems that diagnose and improve the data foundations of frontier AI.

You will :

  • Lead high-impact research on data quality frameworks for post-training LLMs - including techniques for preference consistency, label reliability, annotator calibration, and dataset auditing.
  • Design and implement systems for identifying noisy, low-value, or adversarial data points in human feedback and synthetic comparison datasets.
  • Drive strategy for aligning data collection, curation, and filtering with post-training objectives such as helpfulness, harmlessness, and faithfulness.
  • Collaborate cross-functionally with engineers, alignment researchers, and product leaders to translate research into production-ready pipelines for RLHF and DPO.
  • Mentor and influence junior researchers and engineers working on data-centric evaluation, reward modeling, and benchmark creation.
  • Author foundational tools and metrics that connect supervision data characteristics to downstream LLM behavior and evaluation performance.
  • Publish and present research that advances the field of data quality in LLM post-training, contributing to academic and industry best practices.
  • Desired Capabilities

  • PhD or equivalent experience in machine learning, NLP, or data-centric AI, with a track record of leadership in LLM post-training or data quality research.
  • 5 years of academic or industry experience post-doc
  • Deep expertise in RLHF, preference data pipelines, reward modeling, or evaluation systems.
  • Demonstrated experience designing and scaling data quality infrastructure - from labeling frameworks and validation metrics to automated filtering and dataset optimization.
  • Strong engineering proficiency in Python, PyTorch, and ecosystem tools for large-scale training and evaluation.
  • A proven ability to define, lead, and execute complex research initiatives with clear business and technical impact.
  • Strong communication and collaboration skills, with experience driving strategy across research, engineering, and product teams.
  • Extra Credit

  • Experience with data valuation (e.g. influence functions, Shapley values), active learning, or human-in-the-loop systems.
  • Contributions to open-source tools for dataset analysis, benchmarking, or reward model training.
  • Familiarity with evaluation challenges such as annotation disagreement, subjective labeling, or multilingual feedback alignment.
  • Interest in the long-term implications of data quality for AI safety, governance, and deployment ethics.
  • Perks

    Handshake delivers benefits that help you feel supported-and thrive at work and in life.

    The below benefits are for full-time US employees.

    Ownership : Equity in a fast-growing company

    Financial Wellness : 401(k) match, competitive compensation, financial coaching

    Family Support : Paid parental leave, fertility benefits, parental coaching

    Wellbeing : Medical, dental, and vision, mental health support, $500 wellness stipend

    Growth : $2,000 learning stipend, ongoing development

    Remote & Office : Stipends for home office setup, internet, commuting, and free lunch / gym in our SF office

    Time Off : Flexible PTO, 15 holidays + 2 flex days

    Connection : Team outings & referral bonuses

    Explore our mission, values, and comprehensive US benefits at joinhandshake.com / careers.

    Create a job alert for this search

    Staff AI Research Scientist - Data Quality, Handshake AI • San Francisco, CA, United States

    Related jobs
    • Promoted
    AI Research Scientist / Engineer

    AI Research Scientist / Engineer

    PhizenixMenlo Park, CA, US
    Full-time +1
    AI Research Scientist / Engineer.Menlo Park, CA | On-Site | Full-Time / Direct Hire .Seeking top-tier PhDs (Bay Area preferred) with ICML / ICLR publications in LLM training and inference optimizati...Show moreLast updated: 30+ days ago
    • Promoted
    Staff Applied ML / AI Scientist - Search

    Staff Applied ML / AI Scientist - Search

    Faire IncSan Francisco, CA, United States
    Full-time
    Staff Applied ML / AI Scientist - Search.Faire is an online wholesale marketplace built on the belief that the future is local independent retailers around the globe are doing more revenue than Walma...Show moreLast updated: 1 day ago
    • Promoted
    Senior / Staff AI Research Engineer

    Senior / Staff AI Research Engineer

    Hume AISan Jose, CA, United States
    Full-time
    Hume AI is seeking talented researchers and engineers interested in working with our AI research team to build state-of-the-art speech-language models (SLMs). Our new SLM training method-reinforceme...Show moreLast updated: 9 days ago
    • Promoted
    Staff GenAI Research Scientist

    Staff GenAI Research Scientist

    Databricks Inc.San Francisco, CA, United States
    Full-time
    At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the worl...Show moreLast updated: 30+ days ago
    • Promoted
    Assistant / Associate / Full Project Scientist - Machine Learning & Data Analytics

    Assistant / Associate / Full Project Scientist - Machine Learning & Data Analytics

    InsideHigherEdBerkeley, California, United States
    Full-time
    Assistant / Associate / Full Project Scientist - Machine Learning & Data Analytics Advanced Bioimaging Center Department of Molecular and Cell Biology. The UC academic salary scales set the minimum pa...Show moreLast updated: 30+ days ago
    • Promoted
    Senior / Staff AI Researcher (USA)

    Senior / Staff AI Researcher (USA)

    Articul8Dublin, CA, United States
    Full-time
    At Articul8 AI, we relentlessly pursue excellence and create exceptional AI products that exceed customer expectations.We are a team of dedicated individuals who take pride in our work and strive f...Show moreLast updated: 30+ days ago
    • Promoted
    Senior AI Research Engineer, Handshake AI

    Senior AI Research Engineer, Handshake AI

    HandshakeSan Francisco, CA, United States
    Full-time
    Our three-sided marketplace connects 18 million students and alumni, 1,500+ academic institutions across the U.Europe, and 1 million employers to power how the next generation explores careers, bui...Show moreLast updated: 9 days ago
    • Promoted
    Staff Data Scientist, Curation Ecosystem

    Staff Data Scientist, Curation Ecosystem

    PinterestSan Francisco, CA, United States
    Full-time
    Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we're on a mission to br...Show moreLast updated: 30+ days ago
    • Promoted
    AI & ML - Staff Scientist

    AI & ML - Staff Scientist

    Signify TechnologySan Francisco, CA, United States
    Full-time +1
    Candidates must be authorized to work in the United States.Our client, an emerging start-up, is building a revolutionary recommender system, utilizing Artificial intelligence, for the use of their ...Show moreLast updated: 1 day ago
    • Promoted
    Staff Applied AI / ML Scientist - Search

    Staff Applied AI / ML Scientist - Search

    Faire IncSan Francisco, CA, United States
    Full-time
    Faire is an online wholesale marketplace built on the belief that the future is local - independent retailers around the globe are doing more revenue than Walmart and Amazon combined, but individua...Show moreLast updated: 1 day ago
    • Promoted
    Staff Data Scientist

    Staff Data Scientist

    CharacterRedwood City, CA, United States
    Full-time
    AI is building the future of open-ended interactions between people and intelligent agents.This requires rethinking design, user interfaces, and interactions from first principles.We are seeking a ...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Staff Data Scientist

    Staff Data Scientist

    CervinSan Francisco, CA, United States
    Full-time
    Arkestro’s Predictive Procurement accelerates enterprise spend transformation, using AI and game theory to unlock trapped savings and reduce risk, enabling teams to influence significantly more spe...Show moreLast updated: 1 hour ago
    • Promoted
    Assistant / Associate / Full Project Scientist - Machine Learning & Data Analytics - Advanced Bioimaging Center - Department of Molecular and Cell Biology

    Assistant / Associate / Full Project Scientist - Machine Learning & Data Analytics - Advanced Bioimaging Center - Department of Molecular and Cell Biology

    University of California-BerkeleyBerkeley, CA, United States
    Full-time
    The UC academic salary scales set the minimum pay determined by rank and step at appointment.See the following table for the current salary scale for this position : https : / / www.A reasonable estimat...Show moreLast updated: 30+ days ago
    • Promoted
    Sr. Staff Scientist

    Sr. Staff Scientist

    Bio-Rad LaboratoriesHercules, CA, United States
    Full-time
    In this role, the successful candidate will provide supervisory oversight to the R&D team and technical leadership to project teams, ensuring the effective synthesis and characterization of polymer...Show moreLast updated: 30+ days ago
    • Promoted
    Staff Data Scientist, Strategic Research & Reporting

    Staff Data Scientist, Strategic Research & Reporting

    DiscordSan Francisco, CA, United States
    Full-time
    Discord is used by over 200 million people every month for many different reasons, but there's one thing that nearly everyone does on our platform : . Over 90% of our users play games, spending a comb...Show moreLast updated: 1 day ago
    • Promoted
    AI Research Engineer, Handshake AI

    AI Research Engineer, Handshake AI

    HandshakeSan Francisco, CA, United States
    Full-time
    Handshake is building the future of human data for AI.We partner directly with top AI labs to power large language model (LLM) training and evaluation with high-quality, expert-generated data.As AI...Show moreLast updated: 30+ days ago
    • Promoted
    Staff Forward Deployed Data Scientist / Engineer

    Staff Forward Deployed Data Scientist / Engineer

    Scale AI, Inc.San Francisco, CA, United States
    Full-time
    At Scale AI, we are accelerating the development of artificial intelligence by providing the essential, data-centric infrastructure for building and deploying advanced AI systems.We are looking for...Show moreLast updated: 30+ days ago
    • Promoted
    Staff Applied AI / ML Scientist - Ads

    Staff Applied AI / ML Scientist - Ads

    Faire IncSan Francisco, CA, United States
    Full-time
    Staff Applied AI / ML Scientist - Ads.Faire is an online wholesale marketplace built on the belief that the future is local independent retailers around the globe are doing more revenue than Walmart ...Show moreLast updated: 30+ days ago