Your AI Worked 59 Times. What About the 60th?

What happens when your AI assistant does exactly what you expect 59 times... and then does something completely bananas on the 60th? While teaching a secure coding class, I watched nearly 60 students use the same AI assistant, programming language, and prompts. For 59 of them, everything went as expected. But one student's AI-generated Python code included instructions to suppress linting checks, followed by code that leaked secrets to an external file. The AI hadn't escaped a sandbox. It ...