Engineering Manager, Evaluations & Observability

Engineering Manager, Evaluations & Observability

Join to apply for the Engineering Manager, Evaluations & Observability role at Retool .

Nearly every company in the world runs on custom software for critical operations such as tracking performance metrics, handling customer support workflows, building admin dashboards, and countless other processes you might not have even thought of. Most companies lack adequate resources to invest properly in these tools, leading to old and clunky internal software or, worse, manual and spreadsheet flows.

At Retool, we’re building the first enterprise AppGen platform : software that transforms natural language into production-ready code, integrates directly with business data, and meets the highest standards of security and governance. AI is redefining what it means to build software—and who gets to build it. The definition of “developer” now includes analysts, operators, and domain experts creating solutions directly. As the pool of builders widens, so does the complexity of what they need to build. The opportunity is enormous, but so is the challenge of enabling this larger community to build production‑grade software safely. That means AI that understands real business data, enforces enterprise policies automatically, and empowers teams to create once and reuse everywhere with shared, trusted components.

Over 100 million hours of work has been automated by developers and domain experts using our platform, freeing them to focus on creative problem‑solving and strategic initiatives that drive real business value. The people closest to knowing what needs to be built can now safely create custom solutions within enterprise guardrails. And that’s a mission worth striving for. Let's build the future together!

Why We’re Looking For You

Our engineering leaders are at the forefront of Retool’s product development, bridging the gap between engineering excellence and customer impact. We look for leaders who not only bring strong technical expertise but also the strategic vision to shape Retool’s product direction, balancing day‑to‑day execution with long‑term thinking.

In this role, you’ll lead Retool’s Evaluations & Observability platform. You’ll set the bar for what “good” looks like across our newly‑launched Assist experience, ensuring it works consistently at scale. You’ll build the systems, tools, and culture that let us measure, understand, and improve quality in real time, driving relentless iteration across everything we ship.

You’ll Guide Engineers Focused On

• Evaluation platforms : building the frameworks that let us test and compare performance across LLM providers and model versions.
• Quality systems : defining and enforcing rubrics, metrics, and evaluation loops that answer the hardest question in AI : “Is this actually good?”
• Data curation : managing the datasets that power and test our AI models, sourced from real‑world usage to keep our systems grounded and relevant.
• Search & retrieval quality : owning the retrieval layer that underpins both AI and non‑AI experiences, ensuring results are relevant, accurate, and fast.
• Reusable AI quality infrastructure : creating the building blocks (evaluation tools, pipelines, and feedback systems) that other teams can leverage to maintain quality across Retool’s AI surface area.
• Culture of continuous improvement : embedding a data‑driven approach to AI quality, where experimentation and measurement are the default as we scale our capabilities.

In This Role, You Will

The Skillset You’ll Bring

Salary & Benefits

For candidates based in San Francisco, the pay range for this role is listed below and represents base salary range for non‑commissionable roles or on‑target earnings (OTE) for commissionable roles. This salary range may be inclusive of several career levels at Retool and will be narrowed during the interview process based on a number of factors such as scope and responsibilities, the candidate’s experience and qualifications, and location.

San Francisco : $188,400—$251,900 USD.

Additional compensation may include equity, commission, or bonuses depending on the position offered. Retool provides a comprehensive benefit plan, including medical, dental, vision, and 401(k). Pay and benefits are subject to change at any time, consistent with the terms of any applicable compensation or benefit plans.

#J-18808-Ljbffr