″“Alignment Engineering” vs. “Misalignment Science”” by Edward James Young

<p> There has been much discussion recently around whether a large portion of alignment research is net negative. Without endorsing or refuting them, the basic arguments here are:</p><ul> <li value="1">Prosaic alignment of models is becoming a bottleneck for capabilities.</li><li value="2">Therefore improving the prosaic alignment of models enables faster capabilities advances, which bring us closer to RSI.</li><li value="3">It is unlikely these prosaic alignment methods remain sufficient during the RSI loop, and so this work brings us closer to doom.</li><li value="4">Furthermore, dealing with these more prosaic failures reduces the likelihood of a warning shot of sufficient magnitude to cause a slowdown which would prevent RSI.&nbsp;</li></ul><p> On the basis of this argument, some urge alignment researchers at AGI companies to quit outright. But quit to do what? Missing from this exchange so far has been a discussion of opportunity costs. If you aren’t going to do (technical) work on “Alignment” – either inside or outside of an AGI company – what should you work on? </p><p> In this post, I outline a contrast between “Alignment Engineering” – the dominant model for what “working on alignment” looks like (inside labs, and in the field as a whole) with “Misalignment Science”. I begin by characterising [...]</p> <p>---</p><p><strong>Outline:</strong></p><p>(01:51) "Alignment Engineering"</p><p>(06:22) AI Safety and the ML tradition</p><p>(07:42) Implicit work trials</p><p>(09:36) "Misalignment Science"</p><p>(14:56) Conclusion</p><p>(15:39) Postscript: Iterating ourselves into oblivion</p> <p><i>The original text contained 4 footnotes which were omitted from this narration.</i> </p><p>---</p>

<p><b>First published:</b><br/>

October 5th, 2026 </p>

<p><b>Source:</b><br/>

<a href="https://www.lesswrong.com/posts/FogmcDHA6AdMGukum/alignment-engineering-vs-misalignment-science?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/FogmcDHA6AdMGukum/alignment-engineering-vs-misalignment-science</a> </p>

<p>---</p>

<p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p>