The best AI agents are simpler than you think | Ben Tannyhill, LangChain

<p>Ben Tannyhill is a product manager at LangChain, where he's building LangSmith Engine—an agent that finds and fixes your agent's failures. Engine continuously analyzes your production traces, clusters them into actionable issues, and opens pull requests to fix them. Engine's architecture is a lot like an org chart: a main model delegating to a team of cheaper, faster sub-agents. It launched in public beta at Interrupt 2026, and in this conversation, Ben unpacks why it uses a sandbox as a tool, how the team turned it into a self-improving agent that learns from its own traces, and the hard problem of testing a fix before it ships.</p><p>–</p><p><strong>We also discuss:</strong></p><ul><li>Why Engine is "the agent for agent engineers"</li><li>Making LangSmith agent-native with condensed trace views</li><li>Why the team keeps handing more control to the agent</li><li>Inside Engine's four sub-agents: the screener, verifier, and more</li><li>Giving Engine memory with an agent overview document</li><li>How to keep an always-on agent from blowing the inference budget</li><li>Where Insights, Polly, and Engine are converging</li></ul><p>–</p><p><strong>Timestamps:</strong></p><p>(00:00) Introduction<br />(01:25) LangSmith 101<br />(02:22) Why Engine is "the agent for agent engineers"<br />(03:49) Under the hood: Engine is a deep agent<br />(06:08) Clustering millions of traces with condensed views<br />(10:10) Why the team keeps handing more control to the agent<br />(13:21) Why Engine uses a sandbox as a tool<br />(14:11) Engine's four sub-agents and the org-chart analogy<br />(16:51) Evals for Engine: IssueBench, Harbor, and synthetic environments<br />(23:05) How Engine evolved: from noisy PRs to an issue inbox<br />(25:56) Inside Engine's memory: the agent overview document<br />(29:25) How to keep an always-on agent from blowing the inference budget<br />(30:52) What models Engine uses<br />(31:30) How Engine was rolled out: from Forge to public beta at Interrupt<br />(34:18) Inside the two teams building Engine<br />(35:53) Where Insights, Polly, and Engine are converging<br />(40:06) The missing piece: testing a fix before it ships<br />(42:22) Running a branched agent, and the write-access eval problem<br />(46:35) Using Engine as long-term memory<br />(47:39) Pointing Engine at coding-agent traces<br />(48:49) Running Engine on Engine: the meta self-improvement loop</p><p>–</p><p><strong>References:</strong></p><ul><li><a href="https://www.anthropic.com/" rel="ugc noopener noreferrer" target="_blank">Anthropic</a></li><li><a href="https://chat.langchain.com/" rel="ugc noopener noreferrer" target="_blank">Chat LangChain</a></li><li><a href="https://www.anthropic.com/claude-code" rel="ugc noopener noreferrer" target="_blank">Claude Code</a></li><li><a href="https://www.anthropic.com/claude/haiku" rel="ugc noopener noreferrer" target="_blank">Claude Haiku</a></li><li><a href="https://www.anthropic.com/claude/opus" rel="ugc noopener noreferrer" target="_blank">Claude Opus</a></li><li><a href="https://openai.com/codex/" rel="ugc noopener noreferrer" target="_blank">Codex</a></li><li><a href="https://docs.langchain.com/langsmith/use-the-context-hub" rel="ugc noopener noreferrer" target="_blank">Context Hub</a></li><li><a href="https://www.creditgenie.com/" rel="ugc noopener noreferrer" target="_blank">Credit Genie</a></li><li><a href="https://docs.langchain.com/oss/python/deepagents/overview" rel="ugc noopener noreferrer" target="_blank">Deep Agents</a></li><li><a href="https://gemini.google.com/" rel="ugc noopener noreferrer" target="_blank">Gemini</a></li><li><a href="https://openai.com/index/introducing-gpt-5-5/" rel="ugc noopener noreferrer" target="_blank">GPT-5.5</a></li><li><a href="https://www.harborframework.com/" rel="ugc noopener noreferrer" target="_blank">Harbor</a></li><li><a href="https://hex.tech/" rel="ugc noopener noreferrer" target="_blank">Hex</a></li><li><a href="https://docs.langchain.com/langsmith/insights"