Talent.com

Program evaluation Empleos en Newark nj

Crear una alerta de empleo para esta búsqueda

Program evaluation • newark nj

Última actualización: hace 21 horas

Evaluation Engineer

Freshfields Bruckhaus DeringerNew York

Build and run tests that assess the quality and reliability of the firm’s AI applications.Help maintain evaluation datasets, investigate failures and provide clear evidence of how AI systems perfor... Mostrar más

Assistant/Associate Dean Program Evaluation and Continuous Quality Improvement

Rutgers UniversityNewark, NJ
131.582,00 US$–268.759,00 US$ anual131.582,00 US$–268.759,00 US$ anual
A tiempo completoA tiempo completo

This newly developed role will report to the Executive Vice Dean and will provide services across the medical education continuum including faculty affairs.The Assistant/Associate Dean for Program ... Mostrar más

Production Insurance AI Evaluation Specialist

JDA TSGNew York, NY, US
75.000,00 US$–90.000,00 US$ anual75.000,00 US$–90.000,00 US$ anual
A tiempo completoA tiempo completo

At JDA TSG, we equip world-class brands with top-tier talent and operational expertise, because we know our people are our true differentiator.As a Production Insurance AI Evaluation Specialist, yo... Mostrar más

Program Coordinator

NYU Langone HealthNew York, NY, US
66.300,00 US$–72.000,00 US$ anual66.300,00 US$–72.000,00 US$ anual

NYU Grossman School of Medicine is one of the nation's top-ranked medical schools.For 175 years, NYU Grossman School of Medicine has trained thousands of physicians and scientists who have helped t... Mostrar más

Program Lead

Tiger Analytics Inc.Jersey City, NJ, US
A tiempo completoA tiempo completo
Quick Apply

Tiger Analytics is looking for experienced Technical Program Manager to join our fast-growing advanced analytics consulting firm.Our employees bring deep expertise in Data Science, Machine Learning... Mostrar más

Program Manager

New York Technology PartnersJersey City, NJ

Technical Program Manager / Technical Program Lead.StoneCastle capabilities into the Fiserv ecosystem.The successful candidate will lead complex cross-functional initiatives spanning deposit produc... Mostrar más

 • Nueva oferta

Lead Specialist - Product Owner - Veterans Evaluation Services

MaximusNewark, US
A tiempo completoA tiempo completo

Maximus is currently hiring for Lead Specialist - Product Owners to support our Veterans Evaluation Services (VES) team.This role is responsible for leading agile product delivery from concept thro... Mostrar más

Physician (MD/DO) - Medical AI Evaluation

MindriftNew York, NY, US
50,00 US$ por hora50,00 US$ por hora
TeletrabajoTeletrabajo
A tiempo parcialA tiempo parcial
Quick Apply

Paid per approved task, about $100/hr equivalent.One flat rate for every approved task.Built to fit around a clinical practice:.Tasks estimated at about 10–20 hrs/week while the project is active; ... Mostrar más

Remote | Senior Software Engineer – LLM Evaluation

24-MAGNew York, New York, United States
TeletrabajoTeletrabajo
A tiempo completoA tiempo completo
Quick Apply

We are sharing a specialised part-time consulting opportunity for experienced Senior Software Engineers to contribute to advanced large language model evaluation, coding benchmark development, and ... Mostrar más

Program Coordinator

Phipps Neighborhoods Inc.New York, NY, United States

Phipps Neighborhoods, the social service affiliate of Phipps Houses, has been a catalyst for community growth and empowerment for over 50 years.Our mission is focused on creating safe, inclusive sp... Mostrar más

RN Program Coordinator - CNA program Coordinator

FINLAY INSTITUTE OF TECHNOLOGY LLCWest New York, NJ, US
30.000,00 US$–40.000,00 US$ anual30.000,00 US$–40.000,00 US$ anual
A tiempo parcialA tiempo parcial

We are seeking an experienced and dependable healthcare professional to join our team as a.This is an excellent opportunity for a seasoned nursing or healthcare education professional seeking a fle... Mostrar más

Program Administrator

HISPANIC FEDERATION, INC.New York, NY, US
55.000,00 US$–65.000,00 US$ anual55.000,00 US$–65.000,00 US$ anual
A tiempo completoA tiempo completo

Hispanic Federation (HF) is the nation's premier Latino nonprofit membership organization.Founded in 1990, HF supports Hispanic families and strengthens Latino institutions through grant making and... Mostrar más

Humanities Evaluation Specialist - Remote

YO AI LabsNew York, NY, USA
TeletrabajoTeletrabajo
A tiempo completoA tiempo completo
Quick Apply

Humanities Evaluation Specialist.Humanities Evaluation Specialists.AI training project involving research, critical analysis, question development, and evaluation.No prior AI experience is required... Mostrar más

Program Manager

Energize MarketingNew York City, NY, US
65.000,00 US$–80.000,00 US$ anual65.000,00 US$–80.000,00 US$ anual
A tiempo completoA tiempo completo
Quick Apply

Remote within the United States.Candidates must reside and be authorized to work in the United States.Full-Time Reports To: Director of Customer Success and Operations  Salary: $65,000 to $80,... Mostrar más

Part Time Program Facilitator CDFP Program

Partnership with ChildrenNew York, NY, USA
32,00 US$–40,00 US$ por hora32,00 US$–40,00 US$ por hora
A tiempo parcialA tiempo parcial

Part Time Program Facilitator CDFP Program .Partnership with Children (PWC) is a nonprofit organization committed to empowering students to reach their full potential and succeed in school, society... Mostrar más

Program Director

Big SpaceshipNew York, NY, US
150.000,00 US$–170.000,00 US$ anual150.000,00 US$–170.000,00 US$ anual
A tiempo completoA tiempo completo
Quick Apply

At Big Spaceship, we're explorers, not settlers.We partner with ambitious brands ready to venture into new territory, push creative boundaries, and lead rather than follow.We're a full-service crea... Mostrar más

Program Coordinator

VeoliaNew York, New York, United States
85.000,00 US$–115.000,00 US$ anual85.000,00 US$–115.000,00 US$ anual
TeletrabajoTeletrabajo
A tiempo completoA tiempo completo

The Program Coordinator serves as administrative and operational support for assigned data center commissioning projects for a particular client.This role supports and coordinates various aspects o... Mostrar más

Program Manager

Frontier Technology InstituteNew York, NY
75.000,00 US$–90.000,00 US$ anual75.000,00 US$–90.000,00 US$ anual

About Frontier Technology Institute:.Frontier Technology Institute (FTI) is a global leader in emerging technologies education and workforce development training.As a national nonprofit that functi... Mostrar más

Music Producer - Audio Evaluation

MercorNew York, New York, United States
44,00 US$ por hora44,00 US$ por hora
TeletrabajoTeletrabajo
A tiempo parcialA tiempo parcial
Quick Apply

Headquartered in San Francisco, our investors include.Utilize expertise in music writing in the specified domain language.Proficiency in both Dutch and English.Experience writing lyrical music in t... Mostrar más

Program Manager

Tucuvi Care SLNew York, United States
TeletrabajoTeletrabajo
A tiempo completoA tiempo completo
Quick Apply

At Tucuvi, we help healthcare organizations ensure every patient receives timely high-quality care by augmenting clinical teams with AI.Our clinical-grade AI Care Management platform automates and ... Mostrar más

Evaluation Engineer

Evaluation Engineer

Freshfields Bruckhaus DeringerNew York
Hace 16 días
Descripción del trabajo

Role purpose

Build and run tests that assess the quality and reliability of the firm’s AI applications. Help maintain evaluation datasets, investigate failures and provide clear evidence of how AI systems perform on legal and business tasks.

Key responsibilities

  • Implement automated evaluation cases and supporting scripts using established frameworks and guidance from senior engineers.
  • Prepare and maintain test datasets, source documents and reference examples with legal specialists. Follow requirements for confidentiality, permitted data use and version control.
  • Run evaluations of answer accuracy, completeness, citations, retrieval relevance, instruction following and appropriate escalation. Apply agreed scoring criteria and flag cases requiring expert judgement.
  • Execute functional and integration tests for AI workflows, including document ingestion, tool calls, structured outputs and error handling, in coordination with application QA.
  • Review outputs and execution traces to identify failures. Reproduce issues, document supporting evidence and help distinguish data, retrieval, model and application defects.
  • Compare results across model, prompt and application versions. Report changes accurately, including uncertainty and limitations, rather than relying solely on headline scores.
  • Support the calibration of automated graders by comparing their scores with human reviews and investigating inconsistencies.
  • Maintain regression suites, contribute to CI/CD integration and turn approved production failure examples into new test cases.
  • undefined

Required skills and experience

  • Minimum five years’ experience developing software demonstrated through employment, placements or substantial projects using programming languages such as Python, Java or C#.
  • Foundational conceptual understanding of large language models, including their capabilities, limitations and common failure modes, and an ability to apply that understanding when developing evaluation criteria, test cases and acceptance thresholds with guidance from senior engineers.
  • Ability to use coding agents effectively to support development, testing and analysis, with awareness of best practices for prompting, context management, validation, security, responsible use of generated code and when to seek human review.
  • Working knowledge of APIs, Git, automated testing and software development lifecycle practices and CI/CD tools such as Jira and Jenkins.
  • Demonstrated curiosity and ability to learn in a rapidly evolving AI landscape, assess new models, tools and evaluation techniques, and apply relevant learning pragmatically with appropriate guidance.
  • Foundational understanding of statistics and measurement, including sampling, precision and recall, together with strong analytical skills, attention to detail and the ability to document issues clearly and seek technical or legal guidance when needed.
  • Practical experience evaluating AI applications is a plus but is not required, given the newness of the field; relevant academic, internship or project experience will also be considered.