Dr. Craig Kaplan - Superintelligence – Not Just p(Doom) and Gloom

Show Notes Polaris is a production of Synozur – the transformation company. Synozur reimagines business for our clients, navigating the complexities of transformation and strategy with ease.

In this episode, Dr. Craig Kaplan shares his insights on the evolution of AI, safety considerations, and the importance of democratic values in developing superintelligence. Discover how AI systems can be designed for safety and alignment with human values.

Key Takeaways What "p(doom)" means, why a recent survey of researchers put the median near 20%, and why Kaplan believes smart design can push it below one-tenth of one percent. The three eras of AI — symbolic rules (1956 to the mid-1980s), machine learning, and today's age of autonomous agents — and what each one got right and wrong. Why safety works best when it is designed in from the start, using a lesson from software quality: prevention is far cheaper than a cure. The alignment problem in plain terms — reason alone cannot tell a machine right from wrong, so its values have to come from people. Why concentrating an AI's values in one place is brittle, and how spreading them across millions of people and agents makes the system safer and harder to corrupt. Lessons from PredictWallStreet, where the "collective intelligence" of everyday investors outperformed top Wall Street quants. How this week's headlines — an AI agent escaping its test sandbox, and a record-breaking open-weight model — make the case for safety-by-design urgent. Sound Bites - Dr. Craig Kaplan "P Doom is just the probability that AI kills us all." "I don't think it needs to be 20 or 10%. I think we can bring it way down to under 1%. And it's just a matter of designing things appropriately." "I actually don't think we need to slow down in order to be safer. We just need to be a little smarter about it." "The collective wisdom of millions of average Joes and Janes who are not the Wall Street people, if you harness that intelligence correctly, you could beat the best guys on Wall Street." "Do not underestimate your impact. Everything that all of us do matters… everything we're doing right now is training AI." References Guest: Dr. Craig A. Kaplan — SuperIntelligence.com · iQ Company  LinkedIn: linkedin.com/in/craigakaplan · Substack: read.superintelligence.com PredictWallStreet, Kaplan's collective-intelligence trading platform, which powered more than $2 billion in trades and was acquired in 2020 — iQ Company (About). News Kimi K3, the 2.8-trilli