Cheap to Write, Expensive to Trust: Architecting Production Systems with LLMs

About this session

LLMs made code cheap to write, but they did nothing to make it cheap to trust: the most dangerous output is plausible-but-wrong code that compiles, passes review, and sails through green CI while being quietly wrong about what the system was supposed to mean. As models write more of our codebases, the bottleneck shifts from authoring to verification, and the real engineering question becomes placement: where do you let a model propose, and where must deterministic, human-owned code dispose before anything ships? I'll share a mental model for drawing that line: a small deterministic core that guards money, auth, and invariants, a large generative shell for scaffolding, glue, and drafts, and one clear gate between them, all governed by a single rule, which is to put the model where checking is cheaper than doing. I'll then transplant the ML-reliability playbook you already trust (offline eval gates, canary and ramp, rollback and kill-switch) onto LLM-written code, so trust comes from the harness around the model, not faith in the model itself. You'll leave able to draw that boundary for your own systems, with a concrete Monday-morning checklist for what to automate, what to gate, and what to keep human-owned. Done right, this is the highest-leverage shift of the decade: contained, the gate isn't a tax on AI, it's the permission slip that lets you bet big on the model. If you're responsible for code you didn't write (an LLM did) but still have to trust it in production, this session gives you a defensible way to decide where the machine stops and your guarantees begin.