How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Your AI agent wrote a confident stock report saying 116% of analysts agree.
Nothing in your checks caught it. This is the workshop for fixing that. Laurie Voss (Head of Developer Relations at Arize AI, co-founder of npm) builds a full evals loop on a Claude-powered financial analyst agent: trace it with Arize Phoenix, read the traces, write code evals and LLM-as-a-judge evals, test whether the judge can be trusted, prove a prompt fix with experiments, upgrade from Claude Haiku 4.5 to 5.5 with evidence, and gate it all in CI with npm test. Speaker: Laurie Voss, Head of Developer Relations, Arize AI X: https://x.com/seldo https://arize.com Phoenix: https://phoenix.arize.com 0:00 Intro & setup 2:47 "Looks good to me" isn't a test suite 4:57 Traces vs.
Texten är källans egen beskrivning av publiceringen. Innehållet tillhör AI Engineer.
Mer att läsa
One word for AI agents in 2027
Nate Herk för 3 tim sedan
Tests: chatbots rely on three obscure AI-built voter sites for US midterm answers; Copilot cited the error-prone Ballot Brief in 75% of tests and ChatGPT in 22%
Techmeme för 4 tim sedan
Build an AI Code Reviewer with OpenRouter
OpenRouter för 4 tim sedan
AI:AM: Carbon in the Arena, Mistral's Le Chonk, the Agent-Run Economy, Ferroelectric RAM + New Hacks
The Cognitive Revolution för 6 tim sedan