How to Build Better AI Evals with Claude Code in 5 Steps | Shreya & Hamel

<p>📌 Copy the AI system I use to run a one-person $1M+ business: <a href="https://www.behindthecraft.com">https://www.behindthecraft.com</a></p><p>🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: <a href="https://creatoreconomy.so">https://creatoreconomy.so</a></p><p><br></p><p>Shreya and Hamel teach the industry standard course for AI evaluations taken by 4,500+ students from OpenAI, Google, and more. In our interview, I asked them to give live feedback on the AI evals that I built for my skills and demo how anyone can run evals with ChatGPT/Claude and their free evals skill. Don’t miss this up to date primer on evals from the real experts. 🙂</p><p><br></p><p>P.S., I asked Shreya and Hamel to give my audience 25% off the next cohort of their AI evals course. It has 4.7 stars with 900 reviews which is almost unheard of on Maven. Sign up via the link here: <a href="https://maven.com/parlance-labs/evals?utm_campaign=peteryang&amp;utm_medium=affiliate&amp;utm_source=maven&amp;promoCode=PETERYANG">https://maven.com/parlance-labs/evals?utm_campaign=peteryang&amp;utm_medium=affiliate&amp;utm_source=maven&amp;promoCode=PETERYANG</a></p><p><br></p><p>We talked about:</p><p>(00:00) Why AI evals are completely different now with the latest models</p><p>(02:30) Demo: Live audit of the evals that I built for my AI skills</p><p>(05:14) Top down vs. bottom up evals and why AI sucks at the latter</p><p>(08:16) Spinning up AI agents to grade each eval criteria</p><p>(14:32) Demo: Using Shreyas AI skill to run evals in Claude</p><p>(23:46) How to review output labels to identify failures</p><p>(34:37) How to turn failures into reusable evals</p><p>(43:19) Where human judgement is still needed</p><p><br></p><p>Resources mentioned:</p><p>Free eval skills from Shreya: <a href="https://github.com/ai-evals-course/evals-skills">https://github.com/ai-evals-course/evals-skills</a></p><p>Do automated evals work?: <a href="https://parlance-labs.com/blog/posts/auto-evals/">https://parlance-labs.com/blog/posts/auto-evals/</a></p><p><br></p><p>Thanks to our sponsors:</p><p>Wispr Flow — 4x faster than typing with your voice — <a href="https://ref.wisprflow.ai/peteryang">https://ref.wisprflow.ai/peteryang</a></p><p><br></p><p>📌 Get the takeaways: <a href="https://creatoreconomy.so/p/how-to-build-better-ai-evals-with-claude-code-shreya-hamel">https://creatoreconomy.so/p/how-to-build-better-ai-evals-with-claude-code-shreya-hamel</a></p><p><br></p><p>Where to find Shreya and Hamel:</p><p>Shreya: <a href="https://www.sh-reya.com/">https://www.sh-reya.com/</a></p><p>Hamel: <a href="https://hamel.dev/">https://hamel.dev/</a></p><p><br></p><p>Subscribe to this channel - more interviews coming soon!</p>