arXiv · Oct 21, 2026
Better Policies, Worse Verifier Audits
By Daniel Cieślak, Jacek Łubiński
Verifier stacks increasingly sit inside post-training and test-time compute loops. We study a specific control loop: a component verifier is evaluated on fresh policy outputs against the full success label and refreshed when that aggregate score falls. For a conjunctive target y = F wedge M, its on-policy AUC is (1-q)Aown + qAcross, where q = Pr[F=1 | y=0]. As the policy masters F, q can rise toward one, removing weight from F-failures and rewarding information about M. We call this on-policy audit collapse. We derive a sharp identified set, a complete-erasure threshold, and the exact point at which the audit can prefer a worse component verifier; its population optimum is the full-outcome predictor, not the component oracle. An exact oracle exhibits the collapse, a natural multi-label text control shows inversion can select a head with lower canonical F-AUC at the observed failure mixture, and a prediction-registered ECG study shows that acting on the audit can reduce off-policy F-AUC, transfer the damage to an unseen generator, and increase downstream constraint violations. Preserving own-failure support prevents the damage. Matching label-complexity bounds and a controlled-mixture audit give the corresponding repair: audit the component on its own failures, and keep those failures in the refresh distribution.