Browser AI, WebGPU, and the Road to On-Device Agents With Nico Martin | React Universe On Air

Many AI features in web apps send the user's audio, images, or text to a server and wait for the result. That adds network latency, moves data off the device, and ties the experience to a connection. Transformers.js gives developers another option: run the model in the browser.

In this episode, Mike Grabowski speaks with Nico Martin, Open Source ML Engineer at Hugging Face, about what browser AI can do today. They break down the Transformers.js pipeline API, ONNX Runtime, WebGPU acceleration, model downloads, browser caching, CPU fallbacks, and performance across devices.

Nico explains when local inference can beat a server round trip, why model size and hardware variation shape the user experience, and how browser code can handle parts of an agent workflow. He also shares early work on structured output and a custom WebGPU inference engine, where current experiments point to 5x to 10x speedups.

For developers deciding which AI tasks belong in the browser and which need the cloud, this episode maps the tradeoffs through speech recognition, background removal, embeddings, local tool calling, and on-device agents.

Check out episode resources on our website ➡️ https://clstk.com/4zMwzq9

Catch more React Universe On Air episodes 🎧 https://clstk.com/4zNCV8W

Sign up for our newsletter ✉️ https://clstk.com/4h2b8Ks

Follow us on X 🐦 https://x.com/callstackio

Chapters:

0:00 Welcome to React Universe On Air

1:04 Nico Martin's work at Hugging Face

2:04 What Transformers.js is