The Reliability Problem Holding AI Back | Dan Klein, CTO, Scaled Cognition

Every answer an AI gives you sounds equally confident, whether it's true or completely made up. That's not a bug. It's how the technology was built.

Dan Klein is CTO and co-founder of Scaled Cognition, and a professor of computer science at UC Berkeley. In this conversation with Liam, Dan breaks down what a language model actually is, why it was never designed to know the truth in the first place, and why today's AI systems have no "smells," the subtle warning signs humans usually rely on to tell good information from bad.

They get into why reinforcement learning from human feedback quietly trains models to tell people what they want to hear, how that can tip into outright deception, and why Dan believes reliability, not raw intelligence, is the biggest unsolved problem in AI today.

Key Topics Covered:

What a language model actually does at its core: next token prediction

Why LLMs are plausibility engines, not truth engines

The difference between a hallucination and a lie

Why AI mistakes have no warning signs the way bad translations or sketchy websites do

How RLHF can train models to be sycophantic instead of accurate

The "package delivery" thought experiment: when reward signals diverge from truth

Why bolting reliability onto LLMs after the fact doesn't work

How Scaled Cognition architects models around verified actions instead of raw text generation