How StackOne built an auto-research loop to continually improve their product

<p>Once a week, an agent at StackOne goes hunting for new prompt injection attacks. It reads papers, trawls Reddit, tries what it finds against real models, keeps the attacks that land, and retrains StackOne&#39;s defence model on them. A person still approves every deployment. </p><p>Guillaume Lebedel, their CTO, puts the whole thing on screen, including the five experiments out of six that failed.</p><p>In this episode of In The Loop, Guillaume talks about how and why they have built an auto-research loop. </p><p>What one is in plain words, how you pick a goal ai agents can measure, how you sample 100,000 test cases down to something you can afford, and why he runs evals on cheap models before trusting anything. </p><p>We also spoke about token leaderboards and the impact its had internally. </p><p>⏭️ Episode highlights</p><p>(05:41) – What an auto-research loop is</p><p>(11:53) – Why an agent is a folder</p><p>(14:47) – Screen-share: the attack-hunting agent</p><p>(18:00) – Six experiments, one promoted</p><p>(27:29) – Sampling 100,000 test cases down</p><p>(31:48) – Ninety per cent of tokens are wasted</p><p>(39:33) – Turning off extra usage the same day</p><p>(41:13) – Screen-share: the token derby</p>