AI Engineering: Measure the PRs not the Code

A $600 day of AI coding spend sounds alarming — until you divide it by the PRs shipped. At ~$30/PR, it reframes the entire ROI conversation: the spend isn't the problem, spend with no return is.

This episode comes from a live panel Sam moderated at VentureBeat Transform, featuring Emilie Schario (Kilo Code, acquired by Anaconda the morning of the panel), Amol Jain (Replit), and Jared Go (Symbiotic). The conversation cuts straight to what agentic coding actually looks like in production — not in demos.

On architecture: Kilo routes across 500+ models and processes 10 trillion tokens/month, using expensive frontier models for planning and cheaper open-weight models for execution. Replit built an internal fleet of agents running in isolated cloud VMs — connected to GCP, Datadog, and Slack — that autonomously handled a gnarly multi-day bug and produced a merged PR with zero human intervention. Symbiotic runs tiered monthly token budgets ($150 / $450 / exception) managed directly by engineering managers, not finance.

On org design: code review — not code generation — is now the bottleneck. Replit risk-scores every PR; low-risk ones auto-merge. The panel is aligned that "what does code review mean in 2025" is the defining question for the next 6–12 months.

On the 6-month outlook: open-weight models take majority share of coding tasks, frontier models reserved for architecture decisions; 24–48 hour unattended agent work loops become standard; and model sovereignty (geographic data routing, on-prem isolated environments) becomes a hard enterprise requirement.

---

🎙️ GUESTS: Emilie Schario | Head of Engineering, Kilo Code (acquired by Anaconda) | Amol Jain | VP Engineering, Replit | Jared Go | Co-founder, Symbiotic

🎙️ HOST: Sam Witteveen | VentureBeat

---

00:00 Intro & Cost Per PR framing

00:40 Panel intro: Kilo Code, Symbiotic, Replit

02:20 Do engineers still read code? (Instacart CTO's 97% stat)