Talent.com
Applied AI Researcher, Benchmarking

Applied AI Researcher, Benchmarking

Distyl AINew York, NY, United States
15 hours ago
Job type
  • Full-time
Job description

Distyl AI develops AI native technologies for humans & AI to collaborate to power the operations of the Global Fortune 1000.

In just 24 months, we've rapidly grown to partner with some of the world's largest enterprises-including F100 telecom, healthcare, manufacturing, insurance, and retail companies-delivering multiple AI deployments with $100M+ impact. Our platform, Distillery, along with our team of AI Engineers, Researchers, and Strategists, is pioneering AI-native systems of work, solving the most complex, high-stakes challenges at scale.

Distyl is founded and led by proven leaders from companies like Palantir, Apple, and top national laboratories. We work in deep partnership with OpenAI, jointly going-to-market at the largest enterprises and collaborating evaluating and testing the latest models. Backed by Lightspeed, Khosla, Coatue, industry leaders like Nat Friedman (former GitHub CEO), as well as board members of over 20+ F500s, Distyl is building the future of AI-powered enterprise operations.

What We Are Looking For

At Distyl we're pushing the envelope of AI utilization in enterprise. This requires creative researchers who don't just want to drive incremental improvements on benchmarks or optimize an existing process but instead are looking to creatively redefine how software is used.

Our researchers come from many academic backgrounds but have strong research track records, operate in an AI-native way, and would be bored staying on the rails of a traditional research org.

Key Responsibilities

  • The Benchmarking team defines how progress is measured. Researchers design evaluation frameworks that capture reasoning depth, interaction quality, reliability, and operational impact. They construct benchmarks that reflect real-world complexity. Their systems become the standard by which new architectures, techniques, and releases are judged.
  • Researchers in Benchmarking explore new paradigms for evaluating intelligent systems : adversarial robustness testing, longitudinal performance tracking, and human-in-the-loop assessment. They investigate how metrics shape model behavior and establish rigorous methodologies for quantifying emergent capability. Their insights drive both Distyl's internal research priorities and industry-wide standards.

What We Require

  • Experience Designing and Running Evaluations : You've built or maintained benchmarks, test suites, or experimental frameworks to measure model or system performance.
  • Statistical and Analytical Rigor : You design fair, reproducible experiments and can extract signal from noisy empirical results.
  • Experience Building with Models, Not Just Building Models : We develop intelligent systems using models rather than training or fine-tuning them. Ideal candidates have expertise in compound AI systems, agentic collaboration, and associated techniques (ensembling, ReAct, graph-of-thoughts, etc.).
  • Proven Track Record of Research Results : Whether you've published in top journals, posted amazing work on twitter, or somewhere else we want to see what you've done.
  • Uses AI Every Day : Before you can revolutionize someone else's workflow, you need to revolutionize yours. You should be using tools like ChatGPT, Cursor, and Perplexity to accelerate your workflow.
  • Strong Programming and Data Analysis Skills : While you might not consider yourself a software engineer you need to be able to build prototypes of your ideas and then perform the experiments to prove the effectiveness to a F500 Head of AI.
  • Biases Towards Showing vs Telling : Our customers want to see the power of AI today vs discuss the most elegant idea that will take 5 years to realize.
  • What We Offer

  • An opportunity to advance the cutting edge of LLM research and directly revolutionize work in the enterprise space.
  • Ownership of high-impact research projects, with the autonomy to explore novel approaches and solutions.
  • Access to state-of-the-art AI models, real business problems, and proprietary data sets across a diverse range of real-world industries.
  • Competitive salary and benefits package, including equity options, medical / dental / vision covered at 100% for you and your dependents, 401K plan, and perks such as commuter benefits and lunch provided in office.
  • Be part of a mission-oriented company to create practical adoption during the biggest revolution in human productivity.
  • A collaborative and intellectually stimulating environment that encourages innovation and personal growth.
  • If you are an innovative, ambitious, and driven individual looking to make a difference in the world of AI, we want to hear from you. Apply now to join our team as an Applied AI Researcher and help us shape the future of AI-driven solutions for enterprises across the globe.

    Note : Distyl is a hybrid working environment and requires in office collaboration 3 days a week. We have offices in SF and NYC

    Create a job alert for this search

    Ai Researcher • New York, NY, United States

    Related jobs
    • Promoted
    Applied Researcher I (AI Foundations)

    Applied Researcher I (AI Foundations)

    Capital OneNEW YORK, New York, United States
    Full-time +1
    Applied Researcher I (AI Foundations).At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good. For years, Capital One has been leading the industry in using ma...Show moreLast updated: 30+ days ago
    • Promoted
    Market Research Contributor

    Market Research Contributor

    Prime InsightsValley Stream, NY, US
    Full-time
    Join thousands of members already earning with top-paying surveys and offers.Get started today and enjoy competitive rewards, fast payouts with no waiting periods, and the flexibility to participat...Show moreLast updated: 6 days ago
    • Promoted
    Home-Based Market Research Contributor

    Home-Based Market Research Contributor

    Earn HausHillside, New Jersey, US
    Remote
    Full-time +1
    We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show moreLast updated: 2 days ago
    • Promoted
    Freelance Market Research Contributor

    Freelance Market Research Contributor

    Earn HausFreehold, New Jersey, US
    Full-time +1
    We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show moreLast updated: 2 days ago
    • Promoted
    Remote Market Research Participant

    Remote Market Research Participant

    Earn HausRed Bank, New Jersey, US
    Remote
    Full-time +1
    We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show moreLast updated: 2 days ago
    • Promoted
    Applied Researcher 1

    Applied Researcher 1

    Capital OneNEW YORK, New York, United States
    Full-time +1
    At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...Show moreLast updated: 30+ days ago
    • Promoted
    Applied Researcher II

    Applied Researcher II

    Capital OneNEW YORK, New York, United States
    Full-time +1
    At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...Show moreLast updated: 30+ days ago
    • Promoted
    Applied Researcher I

    Applied Researcher I

    Capital OneNEW YORK, New York, United States
    Full-time +1
    At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...Show moreLast updated: 30+ days ago
    • Promoted
    Applied AI Researcher - Vice President (New York, NY)

    Applied AI Researcher - Vice President (New York, NY)

    The Goldman Sachs GroupNew York, NY, United States
    Full-time +1
    At Goldman Sachs, our Engineers don't just make things - we make things possible.Change the world by connecting people and capital with ideas. Solve the most challenging and pressing engineering pro...Show moreLast updated: 7 days ago
    • Promoted
    Endoscopy Application Analyst II

    Endoscopy Application Analyst II

    RWJBarnabas Health Corporate ServicesOceanport, NJ, US
    Full-time
    Job Title : Application Analyst II.Location : Barnabas Health Corp.Department : EMR Project Capital.The above reflects the anticipated annual salary range for this position if hired to work in New Jer...Show moreLast updated: 6 days ago
    • Promoted
    Applied Researcher II (AI Foundations)

    Applied Researcher II (AI Foundations)

    Capital OneNEW YORK, New York, United States
    Full-time +1
    Applied Researcher II (AI Foundations).At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good. For years, Capital One has been leading the industry in using m...Show moreLast updated: 9 days ago
    • Promoted
    • New!
    Applied AI Researcher, System Self-Construction

    Applied AI Researcher, System Self-Construction

    Distyl AINew York, NY, United States
    Full-time
    Distyl AI develops AI native technologies for humans & AI to collaborate to power the operations of the Global Fortune 1000. In just 24 months, we've rapidly grown to partner with some of the world'...Show moreLast updated: 15 hours ago
    • Promoted
    Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)

    Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)

    Data AnnotationNew Rochelle, New York
    Remote
    Full-time +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...Show moreLast updated: 20 days ago
    • Promoted
    Remote Market Research Contributor

    Remote Market Research Contributor

    Earn HausSalisbury, New York, US
    Remote
    Full-time +1
    We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show moreLast updated: 2 days ago
    • Promoted
    • New!
    UX Researcher, AI & LLM (Drug Discovery)

    UX Researcher, AI & LLM (Drug Discovery)

    SQA SolutionNew York, NY, United States
    Full-time
    About the job UX Researcher, AI & LLM (Drug Discovery).New York, NY (Manhattan, onsite).Please note that at this time we are unable to sponsor employment authorization (both new and transfer).A lea...Show moreLast updated: 15 hours ago
    • Promoted
    Distinguished Applied Researcher

    Distinguished Applied Researcher

    Capital OneNew York City, NY, US
    Full-time +1
    Distinguished Applied Researcher.At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good. For years, Capital One has been leading the industry in using machine...Show moreLast updated: 30+ days ago
    • Promoted
    AI

    AI

    Addivant Usa, LLCStamford, CT, US
    Full-time
    Staff Physician; Nuvance (Norwalk, CT).Assess patient's condition by obtaining med history.Order labs & diagnostic proc req to eval illness. Determine diagnosis & treatment plan.Must possess MD or e...Show moreLast updated: 1 day ago
    • Promoted
    Remote Market Research Interviewer

    Remote Market Research Interviewer

    Earn HausTremont, New York, US
    Remote
    Full-time +1
    We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...Show moreLast updated: 2 days ago