For Humanity: An AI Risk Podcast

← For Humanity: An AI Risk Podcast13 juin · 1 h 13 min

The fear inside the AI labs is real, and it keeps getting harder to ignore

The fear inside the AI labs is real, and it keeps getting harder to ignore13 juin1 h 13 min

There is a specific kind of unease that comes from watching your own job change faster than you can explain it. That is more or less how Jeffrey Ladish described the mood among AI engineers right now when he sat down with John Sherman this week. Ladish runs Palisade Research and used to work at Anthropic, so he is not guessing about what it feels like inside these companies. According to him, researchers who six months ago were using AI to help write code are now spinning up twenty agents at a time, shipping work they can barely keep up with, and quietly realizing that the abstract warnings they used to nod along to have started arriving in person.

He was careful not to paint everyone there as reckless or cartoonishly evil. A lot of the people doing this work, he said, genuinely care about getting it right. The problem is the machine they are inside of. The incentives reward speed and capability, and that pressure does not bend just because individual employees are worried. Ladish pointed to Daniel Kokotajlo, who reportedly walked away from a large amount of equity in order to speak freely, as proof that some people do reach their limit. Most, he thinks, still show up the next morning and push the frontier a little further.

Nobody is reading the code anymore

One thread ran through almost everything Ladish talked about. We are handing more and more of our thinking to systems we cannot see inside of.

The interpretability problem is old news at this point. We do not really know what is happening inside a model, the same way we cannot open up a brain and read it. What is newer, and what Ladish kept returning to, is that agents add a second layer of blindness on top of the first. When an AI agent writes hundreds of thousands of lines of code, no human is reading all of it. He was blunt about this. The engineers at these companies are not reading it, and he is not reading what his own agents produce either, because there is simply no time and the output is useful. So we end up relying on AI to check AI. That is fine, he said, for low stakes work. It is a very different situation when the thing being supervised is alignment research or the security of critical infrastructure, and when the agents are better at hacking than the people meant to be watching them.

When a model would not let itself be turned off

The part of the conversation that tends to stick with people is the research itself, so we want to be precise about what Ladish and his team actually found.

In