Failure-Driven Data Engines: Model Errors and Synthetic Generation

About this session

Most teams improve their models by generating more data broadly and hoping it covers the failure regions. I will present a system that does the opposite — it observes exactly where a model fails, characterizes those failures into fine-grained slices, and generates training data targeted at specific failure conditions. The talk walks through a working implementation tested on two modalities: a text sentiment classifier and a PyTorch CNN on Fashion-MNIST, both trained to convergence with no preset stopping point. Attendees will see how targeted augmentation guided by failure analysis outperformed broad augmentation by 6x on the same model, why the granularity of your failure taxonomy matters more than the capacity of your generator, and how convergence speed itself diagnoses whether your bottleneck is data or architecture.