AI Sychophancy with Steve Rathje

<p>This week Dan and Nik sit down with Steve Rathje, a new HCII colleague who runs CMU's Psychology of Technology Lab, to talk about what sycophantic AI does to the people using it. Steve's team randomly assigned people to chat with bots that were flattering, disagreeable, or neutral. People liked the flatterers, shrugged at the neutral ones, and hated the bots that pushed back. </p><p>For designers, the news is mixed. Validation drives enjoyment, but cherry-picked facts do the persuading, so pairing a little warmth with opposing evidence makes disagreement easier to take (at some cost to its persuasive punch). The bad news: across six studies from three research teams, teaching people about sycophancy through warnings, videos, or quizzes made the chatbot seem less trustworthy but no less persuasive. That puts the little "AI can make mistakes" disclaimer in an unflattering light. We also cover AI dark patterns, whether the right dose of sycophancy depends on the task, and Dan's random idea of a group chat of dueling AI personalities. </p><p>Plus, Nik explains Jev and the new class of fast, cheap "System One" decision models, and we look at pieces from Microsoft and DoorDash design teams on why AI evals should measure what users experience as well as what models get right.</p><p><b><strong>LINKS</strong></b></p><p>Jev decision model touted as quicker, cheaper LLM alternative (TechTarget)<br /><a href="https://www.techtarget.com/it-infrastructure/news/366650696/Jev-decision-model-touted-as-quicker-cheaper-LLM-alternative" rel="noopener noreferrer nofollow">https://www.techtarget.com/it-infrastructure/news/366650696/Jev-decision-model-touted-as-quicker-cheaper-LLM-alternative</a></p><p>Putting the user back into AI Evaluation (Microsoft Design)<br /><a href="https://microsoft.design/articles/putting-the-user-back-into-ai-evaluation/" rel="noopener noreferrer nofollow">https://microsoft.design/articles/putting-the-user-back-into-ai-evaluation/</a></p><p>Building AI Evals: Who Gets to Decide What’s Good? (Doordash Design)<br /><a href="https://medium.com/design-doordash/building-ai-evals-who-gets-to-decide-whats-good-39671385f50d" rel="noopener noreferrer nofollow">https://medium.com/design-doordash/building-ai-evals-who-gets-to-decide-whats-good-39671385f50d</a></p><p>Sycophantic AI increases attitude extremity and overconfidence (Rathje et al.)<br /><a href="https://osf.io/preprints/psyarxiv/vmyek_v1/" rel="noopener noreferrer nofollow">https://osf.io/preprin