Senior AI Engineer APM ExperiencesDatadog • New York City, New York, USA

Senior AI Engineer APM Experiences

Datadog • New York City, New York, USA

7 days ago

Job type

Full-time

Job description

The opportunity

Datadogs APM Experiences team owns the core product experience for Application Performance Monitoring including distributed tracing service representation and more. Were building a new wave of AI-powered capabilities that help customers detect resolve and prevent performance issues this role you will lead endtoend development of LLM- and Agentbased features that can :

Debug and investigate application performance issues down to the root cause as both a developer assistant and a fully autonomous agent
Proactively recommend performance and reliability-based optimizations to prevent the next incident
Automatically create intelligent monitors and SLOs for the most important business flows and critical paths

This is a highly productminded engineering role : youll work from problem discovery and UX all the way to reliable scalable production systems.

What youll do

Shape AI experiences for APM. Design and ship LLM / agentic workflows that analyze traces metrics logs and other telemetry to generate diagnoses explanations and guided fixes.

Own the full loop. Prototype quickly define success metrics and evals run experiments iterate and ultimately productionize for scale and reliability.

Build robust agent systems. Develop tools retrieval and planning strategies and guardrails; manage prompts / evals; design fallbacks and humanintheloop paths.

Integrate with Datadogs platform. Leverage surfaces like Trace Explorer Service Catalog monitors and workflows to deliver endtoend value in the APM UI.

Partner deeply. Collaborate with PM Design and partner teams to build cohesive experiences.

Raise the bar on engineering. Write performant maintainable backend code own services in production and improve reliability for highthroughput lowlatency data systems.

Who you are

Productminded engineer who ships AI to production

4 years building backend or real-time ML systems; you value simplicity correctness and performance

Proven experience delivering LLM / agent features to production (prompting tooling evals safety / guardrails)

Comfortable owning user journeys iterating from prototype alpha GA and measuring impact with clear product metrics

Strong ML / applied science fundamentals

Solid grasp of the ML lifecycle (task definition dataset collection modeling evaluation deployment iteration) and statistics (experiment design confidence intervals)

Experience choosing / modeling the right technique for the job (e.g. anomaly detection ranking / recommendation NLP) and knowing when a heuristic beats a model

Fluency with offline / online evals for AI systems; can build reliable golden sets and automatic regressions

Distributed systems & observability savvy

Experience with microservices performance : tracing latency breakdowns concurrency and resiliency patterns

Proficient in Go Java or Python; strong API / service design; production ops (monitoring alerting oncall rotation)

Nice to have

Handson with distributed tracing stacks (OpenTelemetry / Datadog APM) profilers and logs / metrics pipelines

Exposure to planning / agent frameworks tooluse orchestration RAG and retrieval / indexing for observability data

Familiarity with SLO / SLA practices and incident response

Benefits and Growth :

Get to build tools for software engineers just like yourself. And use the tools we build to accelerate our development.

Have a lot of influence on product direction and impact on the business.

Work with skilled knowledgeable and kind teammates who are happy to teach and learn.

Competitive global benefits.

Continuous professional development.

Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

Required Experience :

Senior IC

Key Skills

APIs,C / C++,Computer Graphics,Go,React,Redux,Node.js,AWS,Library Services,Assembly,GraphQL,High Voltage

Employment Type : Full Time

Experience : years

Vacancy : 1

Create a job alert for this search

Senior Engineer Ai • New York City, New York, USA

Related jobs

Senior AI Engineer

Trafilea • United States Remote, NY, US

Remote

Full-time

Trafilea is a Consumer Tech Platform for Transformative Brand Growth.We’re building the AI Growth Engine that powers the next generation of consumer brands. With over $1B+ in cumulative revenue, 12M...Show more

Last updated: 24 days ago

Senior Technical Service Specialist

Eaton • Tinton Falls, NJ, United States

Full-time

Eaton’s FFD Fluid Filtration Division division is currently seeking a Senior Technical Service Specialist.This is a Full-Time role, working M-F Daylight hours. This position has the ability to be Hy...Show more

Last updated: 19 days ago • Promoted

Senior AI Integration Engineer

Openkyber • NY, United States

Full-time

Quick Apply

Note : Interview 2 video interview and 1 F2F interview at Natick.Expenses are covered for F2F interview.Provide end-to-end architectural leadership for enterprise applications and modernization init...Show more

Last updated: 5 days ago

Senior Software Engineer, AI

Imprint • New York, NY, United States

Full-time

Imprint is reimagining co-branded credit cards & financial products to be smarter, more rewarding, and truly brand-first. We partner with companies like Rakuten, Booking.H-E-B, Fetch, and Brooks Bro...Show more

Last updated: 30+ days ago • Promoted

Remote Epic HIM Analyst

Insight Global • Oceanport, NJ, United States

Remote

Full-time

Defining Systems Requirements : .Collaborating to understand and execute the Epic application architecture and integration. Serving as a liaison between end-u ' workflow needs and Epic implementation ...Show more

Last updated: 19 days ago • Promoted

Endoscopy Application Analyst II

RWJBarnabas Health Corporate Services • Oceanport, NJ, United States

Full-time

Job Title : Application Analyst II.Location : Barnabas Health Corp.Department : EMR Project Capital.The above reflects the anticipated annual salary range for this position if hired to work in New Jer...Show more

Last updated: 26 days ago • Promoted

Senior AI Engineer

Resonance • New York, NY, US

Full-time

Quick Apply

Resonance is a technology company building a more sustainable and valuable fashion industry for designers, brands, manufacturers, consumers, and the planet. The company’s AI-powered operating system...Show more

Last updated: 30+ days ago

Project Engineer

Equiliem • Asbury Park, NJ, US

Full-time

Position Overview : The Project Engineer position offers a dynamic and self-motivated individual the unique opportunity to be part of a rapidly growing business in a rewarding field.The position off...Show more

Last updated: 21 days ago • Promoted

Senior AI Research Engineer, Handshake AI

Handshake • New York, NY, United States

Full-time

Our three-sided marketplace connects 18 million students and alumni, 1,500+ academic institutions across the U.Europe, and 1 million employers to power how the next generation explores careers, bui...Show more

Last updated: 10 days ago • Promoted

Principal Cyber Security Engineer

Hatch Global Search • Tinton Falls, NJ, United States

Full-time

Principal Cyber Security Engineer.Principal Cyber Security Engineer.Monmouth County, NJ based client.This senior-level position requires deep technical knowledge and advanced problem-solving skills...Show more

Last updated: 16 days ago • Promoted

Agentic and Gen AI Architect - Hybrid

Cognizant • Fair Lawn, NJ, US

Full-time

Gen AI and Agentic AI Architect.Teaneck, NJ or Plano, TX (Hybrid – 2 to 3 days per week in office).We are seeking a visionary and pragmatic AI Architect to lead the design and implementation of Gen...Show more

Last updated: 14 days ago • Promoted

AI Engineer

Fay • New York, NY, United States

Full-time

Fay is a 3-sided AI platform redefining preventative care with a b2b2c business in a box.We’re one of the fastest growing companies in tech and the fastest growing company in wellness history.We co...Show more

Last updated: 13 days ago • Promoted

SAP S4 Hana Dev (BC)

Clark Davis Associates • Eatontown, NJ, United States

Full-time

Provide ongoing support and maintenance for S / 4 Hana Utilities solutions, troubleshooting issues, resolving incidents, and implementing enhancements as required. Analyze Business Requirements : Colla...Show more

Last updated: 16 days ago • Promoted

Engineer

Quality Talent Group • Clarkstown, New York, United States

Full-time

Quick Apply

Our client is a leading force in advancing safer, smarter AI technology.Their work has been featured in.They’ve built a global community of expert contributors and have already paid out more ...Show more

Last updated: 2 days ago

Senior AI Engineer

Tiger Analytics • Jersey City, NJ, US

Full-time

Quick Apply

Tiger Analytics is seeking a highly skilled.In this role, you will design, develop, and implement cutting-edge AI solutions that drive business value for our clients. You will leverage advanced mach...Show more

Last updated: 30+ days ago

Full Time, Part Time or Per Diem Anesthesiologist

Allied Digestive Health • West Long Branch, US

Full-time +1

Welcome to Allied Digestive Health! Allied Digestive Health is one of the largest integrated networks of gastroenterology care centers in the nation with over 200 providers and 60 locations through...Show more

Last updated: 30+ days ago • Promoted

Senior Application Security Engineer

Yantran LLC • Middletown, NJ, United States

Full-time

Senior Application Security Engineer.Location : Middletown, NJ (F2F Required, Onsite from Day.We are looking for a Senior Application Security Engineer to join our growing team and play a hands-on r...Show more

Last updated: 27 days ago • Promoted

Applied AI Engineer, Enterprise GenAI

Scale AI, Inc. • New York, NY, United States

Full-time

AI is becoming vitally important in every function of our society.At Scale, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, ...Show more

Last updated: 30+ days ago • Promoted