Principal or Staff AI / ML Engineer

Want to work on the next wave of AI?

Join Wallaroo.AI and help us build the next generation of enterprise AI inference software!

Wallaroo.AI is on the hunt for a Principal or Staff AI / ML Engineer to lead the charge in pushing the boundaries of state-of-the-art inference capabilities on the latest hardware.

We're looking for an expert with a passion for LLM deployment, inference, and lifecycle, coupled with a deep understanding of LLM architectures and cutting-edge generative AI serving runtimes.

Our mission at Wallaroo.AI is to enable inferencing on "any hardware," so if you thrive on mastering new technologies, diving deep, and extracting maximum performance, you'll feel right at home.

Wallaroo.AI is a Microsoft-backed startup at the forefront of AI, developing a powerful software platform that empowers AI teams to deploy AI inference at scale to any cloud, edge or on-prem environment with unparalleled speed and efficiency.

We have marquee enterprise customers and technology partners, including some of the most globally recognized brands.

We’re poised for explosive growth in 2026 fueled by our go-to-market with leading AI hardware providers.

This is your chance to be part of something truly transformative.

We’ve been fully distributed and remote-first since before it was cool!

This means we seek exceptional communicators who are self-motivated and adept at both independent and collaborative work.

We also need individuals who thrive in dynamic, fast-paced environments.

Successful Wallaroo.AI team members are comfortable experimenting with fresh ideas, taking initiative without constant oversight, adapting swiftly as priorities evolve, and bringing clarity to complex, emerging challenges.

Prior startup experience or a similar high-velocity environment is a significant advantage.

Responsibilities :

• At Wallaroo.AI, building a great product is a team endeavor.
• We expect active participation in discussions, proactive communication, and continuous feedback.
• You'll partner closely with customers and hardware teams to solve problems and build capabilities.
• As an experienced engineer, you'll negotiate tradeoffs with product managers, break down feature requests into incremental, high-quality deliverables, and mentor less experienced team members.
• You'll also propose process and workflow improvements, and confidently advocate for your perspective while thoughtfully listening to others.

Required Skills / Abilities (at least 4) :

Strong experience with vLLM, SGLang, and other modern serving frameworks Experience developing high-scale production ML pipelines A strong foundation in deep learning architectures, especially the Transformer architecture, including experience with NLP tasks like text generation, summarization, and sentiment analysis using commonly available LLM APIs and SDKs (ie, OpenAI, Anthropic, Mistral, etc) Desired Skills / Abilities :

Resume Review :

There are a mix of technical and behavioral questions but no technical exercises.  Technical Interview :

Behavioral Interview :

Hiring Manager Interview :