S 933: The AI Medical Diagnosis Problem: Why Accuracy Falls Apart in Real Conversations

<p>AI can score highly on medical exams, identify conditions in written cases, and still perform poorly when real people use it for their health. In this episode, I break down new research that tested what happens when members of the public use AI tools to interpret medical scenarios and decide where to seek care. The results point to a problem that has less to do with the model's knowledge and more to do with the human interaction around it. </p><p><br /></p><p>Topics discussed: </p><p>- The Hemorrhage Case Study</p><p>- The Nature Medicine Study Design</p><p>- 95% to 35% Accuracy Drop</p><p>- The Control Group Outperformed AI</p><p>- Where the AI-Human Handoff Fails</p><p>- Reading Confidence as Competence</p><p>- The 1 in 6 Adults Using Health Chatbots </p><p><br /></p><p>---------- </p><p><a href="http://www.metabolismschool.com/" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠My Live Program for Coaches: The Functional Nutrition and Metabolism Specialization ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a></p><p><a href="http://www.metabolismschool.com/" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠www.metabolismschool.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a></p><p>---------- <br />[Free] Metabolism School 101: The Video Series<br /><a href="http://www.metabolismschool.com/metabolism-101" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠http://www.metabolismschool.com/metabolism-101⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a><br />----------<br />Subscribe to My Youtube Channel: <br /><a href="https://youtube.com/@sammillerscience?si=s1jcR6Im4GDHbw_1" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://youtube.com/@sammillerscience?si=s1jcR6Im4GDHbw_1⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a><br />----------<br /><a href="https://metabolismmadesimple.com/" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Grab a Copy of My New Book - Metabolism Made Simple⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a><br />----------<br /> Stay Connected: <br /><a href="https://www.instagram.com/sammillerscience/" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Instagram: @sammillerscience⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a><br /><a href="https://www.youtube.com/channel/UCKi5AgCKH37irVYvlIYC-vA" target="_blank" rel="ugc noopener noreferrer">⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Youtube: SamMillerScience⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠</a> <br /><a href="https://www.facebook.com/groups/nutritioncoachi