How I AI

I’ve been testing every major frontier model release since the start of the year, and when Anthropic dropped Sonnet 5, I wanted more than a vibe check. I got tired of one-off tests I couldn’t repeat or compare over time, so I built something better: the How I AI Bench, a repeatable eval harness I constructed live using Claude Code while recording this episode. I ran Sonnet 5 blind against four other frontier models (Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro) across PRD quality, prototype generation, agentic task completion, and agent personality. The results were not what I expected.

What you’ll learn:
What Anthropic claims Sonnet 5 improves over Sonnet 4.6, and where the benchmark data actually backs that up
How I built the How I AI Bench in under 45 minutes using Claude Code, starting from my own stored session history
Why I combined human vibe scoring (70%) with LLM as judge scoring (30%) instead of trusting either alone
How to set up a local HTML scoring page so you can rate AI outputs on gut feel and export those scores as JSON
Which model I recommend for PRDs, which for complex prototypes, and which for chatting with an agent daily
—
Brought to you by:
Runway—The creative AI platform for images, video and more
Hyperagent—Deploy fleets of agents that handle real work
—
In this episode, we cover:
(00:00) Sonnet 5 is out
(01:55) What Anthropic claims
(04:02) Why I’m done with one-off vibe checks
(05:05) Building the How I AI Bench live with Claude Code
(07:42) The scoring system
(10:43) Agent voice eval
(11:57) Quick recap
(13:58) Results: The How I AI index leaderboard
(21:21) What I’m improving for the next run
(22:16) Generating a Claire-weighted index
(23:53) Model-by-task recommendations
—
Tools referenced:
• Claude Sonnet 5: https://www.anthropic.com/news/claude-sonnet-5
• Claude Opus 4.8: https://www.anthropic.com/news/claude-opus-4-8
• GPT-5.5 (OpenAI): https://openai.com/index/introducing-gpt-5-5/
• Gemini 3 Pro (Google DeepMind): https://deepmind.google/models/gemini/pro/
• Cursor: https://www.cursor.com/
—
Other references:
• SWE-bench Pro (agentic coding benchmark referenced): https://www.swebench.com/
—
Where to find Claire Vo:
ChatPRD: https://www.chatprd.ai/
Website: https://clairevo.com/
LinkedIn: https://www.linkedin.com/in/clairevo/
X: https://x.com/clairevo
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Eddie Kim is the co-founder and CTO of the payroll and HR platform Gusto, which just crossed $1 billion in revenue and serves more than 500,000 small businesses. Recently he did something most CTOs don’t: he went back to writing code. With three other engineers and one designer, Eddie built Gusto Cofounder, a net-new AI product, from zero code to a tier-one launch in 10 weeks. He walks through how that team actually worked, why they threw out nearly every process, and how anyone can copy the approach.

What you’ll learn:
The trash-can method: how to write, review, and delete a full PR as a product decision instead of a planning doc
The two-tool agent stack behind Gusto Cofounder
The exact “perma-Zoom” setup that replaced standups, retros, and Slack threads for 10 weeks
How a designer with no engineering background hit the 94th percentile for shipping code
The eval-first workflow Eddie uses to fix real customer bugs with Claude Code
How a non-technical leader can prototype an idea to win buy-in, then carry it all the way to production-quality code
—
Brought to you by:
Magic Patterns—Prototypes that look like your product
Jira Product Discovery—Prioritize with insights, build with confidence
—
In this episode, we cover:
(00:00) Intro: five people, 10 weeks
(02:38) The origins of Cofounder
(08:32) Inside the 10-week build process
(12:50) Building with no PMs
(14:38) The “trash can” method
(17:15) The stack architecture
(19:10) Shipping to production from day one
(22:03) How a designer became a top engineer
(29:05) Demo: Cofounder over text and Slack
(31:45) Demo: running a real payroll
(36:26) Live coding with evals in Claude Code
(39:39) Recap: prototype, small team, permission
(43:17) Lightning round
(48:44) Where to find Eddie and Cofounder
—
Tools referenced:
• Gusto Cofounder (early access/waitlist): https://gusto.com/cofounder
• Claude Code (Anthropic): https://claude.ai/code
• Cloudflare Workers: https://workers.cloudflare.com/
• Vercel AI SDK: https://sdk.vercel.ai/
• DX (engineering analytics): https://getdx.com/
• Wispr Flow (voice-to-text): https://wisprflow.ai
• OpenClaw: https://openclaw.ai/
—
Other references:
• Gusto (the main product, “Gusto Classic”): https://gusto.com
• Mindbody (referenced as customer data source): https://www.mindbodyonline.com/
—
Where to find Eddie Kim:
LinkedIn: https://www.linkedin.com/in/edawerd/
—
Where to find Claire Vo:
ChatPRD: https://www.chatprd.ai/
Website: https://clairevo.com/
LinkedIn: https://www.linkedin.com/in/clairevo/
X: https://x.com/clairevo
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I put GLM 5.2, the open-weight coding model from Z.AI, through four real tasks inside my actual codebase: a codebase architecture audit, a UI redesign, and a 45-minute autonomous bug-hunting session pulling from Sentry and Vercel logs. Total cost: $3.36 for roughly 6 million tokens, a prioritized bug-fix dashboard I’m actually shipping from, and a landing page redesign that matched Chat PRD’s design system on the first try.

What you’ll learn:
What “open-weight” actually means and why it matters for cost and vendor independence
How to connect GLM 5.2 to Cursor and Claude Code
How it performs on codebase exploration and autonomous architecture summarization in a real production Next.js app
Whether GLM 5.2 can match an existing design system
How the model handles a 45-minute long-running autonomous task
Where GLM 5.2 stumbled 
The actual cost breakdown
—
Brought to you by:
Mercury—Radically different banking loved by over 300K entrepreneurs
—
In this episode, we cover:
(00:00) What open-weight models are and why GLM 5.2 is worth testing
(01:38) GLM 5.2 model overview
(04:02) Capabilities and benchmark results
(06:02) How to set up GLM 5.2 in Cursor
(08:37) How to set up GLM 5.2 in Claude Code
(11:04) Live test 1: codebase exploration and architecture audit on ChatPRD
(12:43) Live test 2: generating an HTML architecture and roadmap page
(16:37) Live test 3: redesigning the How I AI landing page in Cursor
(20:57) Live test 4: 45-minute autonomous task, pulling Sentry errors and Vercel logs
(22:35) Where it struggled
(23:49) My verdict on the output
(25:23) Cost breakdown
—
Tools referenced:
z.ai: https://z.ai
GLM 5.2: https://z.ai/blog/glm-5.2
OpenRouter: https://openrouter.ai
Cursor: https://cursor.com
Claude Code: https://docs.anthropic.com/en/docs/claude-code
Sentry: https://sentry.io
Vercel: https://vercel.com

—
Other references:
SWE-Bench Pro leaderboard (coding benchmark scores referenced in episode): https://www.swebench.com
Frontier Suite and Post-Train Bench (additional benchmarks cited): https://scale.com/leaderboard
Use Claude Code with OpenRouter: https://openrouter.ai/docs/cookbook/coding-agents/claude-code-integration
—
Where to find Claire Vo:
ChatPRD: https://www.chatprd.ai/
Website: https://clairevo.com/
LinkedIn: https://www.linkedin.com/in/clairevo/
X: https://x.com/clairevo
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Brian Grinstead is a distinguished engineer at Mozilla, where he’s worked on Firefox and the web platform since 2013 (he joined to help launch Firefox DevTools). Recently he and his team pointed an agentic bug-finding pipeline at Firefox—a codebase with tens of thousands of files and tens of millions of lines of code—and shipped a record month of security fixes. The viral chart everyone saw gave the credit to Anthropic’s new Mythos model. Brian’s take is that the harness and pipeline did just as much of the work, and he walks through exactly how it runs and how anyone can build a starter version.

What you’ll learn:
How to build a basic bug-finding harness by running Claude Code or Codex with one prompt and the -p flag, no SDK required
Why pointing an agent at a whole codebase fails, and how an LLM judge can score and rank files before you spend any compute
How a verifier subagent kills false positives by catching the agent when it cheats
The goal-loop pattern: give an agent a tightly scoped problem, a clear pass/fail signal, and let it retry far past the point a human would quit
Why teams that already invested in fuzzing, CI, and dev tooling are so far ahead
How to weigh model versus harness, and why Brian splits the credit close to 50-50
How a non-engineer can reuse the same score, verify, and fix the loop for design quality, conversion rate, or tech debt
Why AI-generated patches still can’t ship on their own, and where humans stay in the loop
—
Brought to you by:
WorkOS—Make your app enterprise-ready today
Metaview—The agentic recruiting platform for winning teams
—
In this episode, we cover:
(00:00) Introduction to Brian Grinstead
(02:43) The viral chart: Firefox Security Bug Fixes by Month
(05:32) How the custom harness works
(10:22) Goal loops and guardrails
(14:45) How they built it
(16:55) Real bugs, including a 15-year-old one
(23:00) Open-sourcing it
(26:26) Why humans still review every fix
(32:30) Live demo and prioritizing files
(40:18) Mobilizing the team and recap
(42:33) Lightning round
—
Tools referenced:
• Claude Code: https://claude.ai/code
• Claude Agent SDK: https://code.claude.com/docs/en/agent-sdk/overview
• Codex: https://openai.com/index/openai-codex/
• OpenAI Agent SDK: https://developers.openai.com/api/docs/guides/agents
• VS Code: https://code.visualstudio.com/
• Docker: https://www.docker.com/
• Firefox: https://www.mozilla.org/firefox/
• Address Sanitizer: https://github.com/google/sanitizers
• RLBox: https://rlbox.dev/
—
Other references:
• Mozilla Bug Bounty Program: https://www.mozilla.org/security/bug-bounty/
• Mozilla GitHub: https://github.com/mozilla
—
Where to find Brian Grinstead:
LinkedIn: https://www.linkedin.com/in/bgrins/
GitHub: https://github.com/bgrins
—
Where to find Claire Vo:
ChatPRD: https://www.chatprd.ai/
Website: https://clairevo.com/
LinkedIn: https://www.linkedin.com/in/clairevo/
X: https://x.com/clairevo
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I break down every loop type from scratch—what a heartbeat, cron, hook, and goal loop actually are, when each one fits, and the five things any effective loop needs before it touches production. Then I build two live loops: a daily aging-PR reviewer in Claude Code that schedules itself at 10:15 a.m. and spins off its own subagents, and a weekly skills-identification loop in Codex that spawns goal-based subagents to validate its own output in real time.

What you’ll learn:
The plain-English definition of a loop—and why it’s just an automated prompt, not a scary new paradigm
The four loop types (heartbeat, cron, hook, and goal) and when each one actually fits your workflow
How to think about loop design using the “onboarding an employee” mental model
The five things every effective loop needs: work trees, skills, plugins/connectors, subagents, and state tracking
How to build a scheduled PR-review routine in Claude Code that babysits aging PRs and alerts your team
How to set up a weekly skills-identification automation in Codex that spawns its own validating subagents
Why goal-based loops are the hardest to write well—and where most people burn tokens for nothing
The two warning signs that your loop is going to get expensive before it gets useful
—
Brought to you by:
WorkOS—Make your app enterprise-ready today
Runway—The creative AI platform for images, video, and more
—
In this episode, we cover:
(00:00) Prompts are out and loops are in
(02:30) Defining a loop
(03:03) The four ways to automate a prompt: heartbeat, cron, hooks, and goals
(06:03) Five things every effective loop needs
(09:26) The “onboarding an employee” framework for designing loops
(11:58) Live build #1: Daily aging PR loop in Claude Code
(17:08) Subagents inside loops
(19:00) Live build #2: Weekly skills identification loop in Codex
(22:57) Watching subagents spin up in real time
(25:28) Warning signals around loops
(27:31) What listeners are doing with loops
—
Tools referenced:
• Claude Code: https://claude.ai/code
• Codex: https://chatgpt.com/codex
• OpenClaw: https://openclaw.ai/
—
Other references:
• Claire’s article “Why OpenClaw Feels Alive Even Though It’s Not”: https://x.com/clairevo/article/2017741569521271175
• Addy Osmani’s article on loop engineering: https://addyosmani.com/blog/loop-engineering/
• Using Goals in Codex: https://developers.openai.com/cookbook/examples/codex/using_goals_in_codex
—
Where to find Claire Vo:
ChatPRD: https://www.chatprd.ai/
Website: https://clairevo.com/
LinkedIn: https://www.linkedin.com/in/clairevo/
X: https://x.com/clairevo
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.