AI R&D Automation and the Intelligence Explosion Question

This episode examines a policy paper asking what happens if AI automates AI research itself, setting off an "intelligence explosion" that compresses years of progress into months. It reviews the evidence that automation is already underway, including the AI share of approved code at one lab rising from low single digits to over 80 percent, autonomous R&D work growing from 1 to 26 percent, and task-length horizons doubling about every three months. It also covers the caveats on that evidence, since coding benchmarks don't necessarily translate to research ability. The hosts explain the paper's feedback-loop model, in which a ratio of labor returns to idea-finding difficulty decides whether progress fades, holds steady or accelerates. They also cover the four frictions that could stop the loop (diminishing returns, compute limits, hard-to-automate research taste, and slow training runs) and the stakes of loss of control and eroded checks on power. Listeners get a clear walkthrough of what the argument claims and what it doesn't: it argues for preparation, not that an explosion is certain.

Sources:

1. What if automating AI R&D triggers an intelligence explosion? — Alan Chan, Christoph Winter, Andrew Barto, Jakub Pachocki, Geoffrey Hinton, Eric Horvitz, Yoshua Bengio, Dawn Song, Jack Clark, Hilary Greaves, Anton Korinek, Samuel Hammond, Thore Graepel, Ben Bariach, Philip H. S. Torr, Sheila A. McIlraith, Jeff Clune, Sam Manning, Girish Sastry, Tom Davidson, Daniel Eth, Sören Mindermann, 2026

http://arxiv.org/abs/2609.36054

2. Speculations Concerning the First Ultraintelligent Machine — I.J. Good, 1965

https://scholar.google.com/scholar?q=Speculations+Concerning+the+First+Ultraintelligent+Machine

3. Intelligence Explosion Microeconomics — Eliezer Yudkowsky, 2013

https://scholar.google.com/scholar?q=Intelligence+Explosion+Microeconomics

4. Are Ideas Getting Harder to Find? — Nicholas Bloom, Charles I. Jones, John Van Reenen, Michael Webb, 2020

https://scholar.google.com/scholar?q=Are+Ideas+Getting+Harder+to+Find%3F

5. Algorithmic progress in language models — Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carson Guo, David Atkinson, Neil Thompson, Jaime Sevilla (Epoch AI), 2024

https://scholar.google.com/scholar?q=Algorithmic+progress+in+language+models