ASR vs Multimodal LLMs: What Actually Won

A year ago, the prediction was that dedicated ASR models would go extinct — why build a transcription model and a second model to clean up its output when a multimodal LLM could do both from raw audio? Two years of evidence says otherwise. The top of the Open ASR Leaderboard is still owned by Conformer-encoder-plus-LLM-decoder systems on accuracy, but a 0.6B TDT model sits four tenths of a point behind at roughly eight times the speed, and a new third layer has appeared: specialized post-processing models like Superwhisper's S1-mini that turn raw transcripts into formatted text. This episode works through the three-layer stack, why the latency gap is an architectural fork rather than a maturity gap, why word error rate is under active attack as a metric, and why no benchmark yet compares audio-to-formatted-text against ASR-plus-normalization head to head.

Key sources:

• Open ASR Leaderboard paper (v2, 2025-10-08 data) — https://arxiv.org/html/2510.06961v2

• NVIDIA Parakeet TDT 0.6B v3 model card — https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3

• S1-mini model card — https://huggingface.co/superwhisper/s1-mini

Episode 6028 — myweirdprompts.com/n/6028 (permanent short link: myweirdprompts.com/351585)