Inference Software Engineer

About Etched

Etched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200. With Etched ASICs, you can build products that would be impossible with GPUs, like real-time video generation models and extremely deep & parallel chain-of-thought reasoning agents.

Key responsibilities

• Contribute to the architecture and design of the Sohu host software stack
• Implement high-performance, modular code across the complete Etched software stack, consisting of a mix of Rust, C++ and Python.
• Interface with firmware and drivers teams delivering highest-performance HW / SW stack.
• Work with AI model researchers and product-facing teams building out the Etched serving front-end.

Representative projects

You may be a good fit if you have

Strong candidates may also have

Benefits

How we're different

Etched believes in the Bitter Lesson. We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware. Larger and larger training runs encourage companies to consolidate around fewer model architectures, which creates a market for single-model ASICs.

We are a fully in-person team in Cupertino, and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.