Your Agent's Output Is a Hypothesis

About this session

Getting AI agents to act is mostly solved but trusting what they did isn't. Agents fail in a way software doesn't: confidently, returning a wrong answer that looks right. I build AI agents inside an experimentation platform, and the verification method everyone else is building from scratch was already sitting in my product: the A/B test. An agent's output is a hypothesis, and a controlled experiment is how you learn whether it's true. Unlike an AI grading an AI, a measured outcome can't be talked into a wrong answer. This talk is what building trustworthy agents taught me: treat every output as a hypothesis, match verification rigor to the cost of being wrong, and let measured outcomes, not confidence, close the loop.