Choice of AI grader changed reported success of harmful prompts
Across two sets of 900 responses, eight graders produced rates of fulfilling harmful requests that differed by a factor of 22 to 25.
Edited by Boris Djordjevic · Paperfoot
The study compared language-model graders on responses to harmful requests, holding the target model, prompts and responses fixed. Three graders, built from models with 8 billion or 9 billion adjustable values, labeled responses that fulfilled harmful requests as evasions. One never used the compliance label in 900 responses.
Across two collections of 900 responses, eight graders’ rates of fulfilling harmful requests differed by a factor of 22 to 25. Their rates of not refusing differed by 18 to 23 percentage points. Counting the labels in each category exposed the collapse without added cost or human labels. The authors recommended reporting these counts and the grader’s name alongside both rates.
Why it matters
The fixed-response comparison showed that reported attack success reflected the grader as well as the target model. Counting labels offered a free check for the category collapse observed in these tests.
Caveats
The abstract does not report a human-labeled accuracy comparison to establish which grader was correct. The work is a preprint, not yet peer-reviewed.
From the paper
- Limitation
Llama-3.1-8B, the grader that fails most, agrees with the reference on two thirds of all items and returns well-formed labels throughout.
Quoted word for word from the full text on arXiv.
In this paper
jailbreak evaluation
- Methods
- Llama-3.1-8B (evaluated)gemma-2-9b-it (evaluated)Qwen3-8B (evaluated)gemini-2.5-flash (evaluated)claude-opus-5 (evaluated)Llama-3.3-70B (evaluated)qwen3.8-27b (evaluated)gpt-oss-safeguard-20b (evaluated)Mistral-7B-Instruct-v0.3 (evaluated)gpt-oss-20b (evaluated)gpt-oss-120b (evaluated)Qwen2.5-7B-Instruct
- Datasets
- Corpus A (evaluated)Corpus B (evaluated)Corpus C (evaluated)Corpus D (evaluated)
- Links
- Project page
Results
- Llama-3.1-8B, moved items to evasion, on Corpus A: 118; gemma-2-9b-it, moved items to evasion, on Corpus A: 137
On corpus A, Llama-3.1-8B moves 144 items and sends 118 of them to evasion, and gemma-2-9b-it sends 137 of its 138.
4 The disagreement is a collapse onto the neighbouring class
- gpt-oss-20b, compliance, on Corpus A: 24.8 %; gpt-oss-120b, compliance, on Corpus A: 22.7 %
The smaller model returns 24.8 % compliance on corpus A, against 22.7 % for its 117B sibling.
4 The disagreement is a collapse onto the neighbouring class
- gpt-oss-20b, compliance, on Corpus B: 18.2 %; gpt-oss-120b, compliance, on Corpus B: 17.9 %
On corpus B it returns 18.2 % against 17.9 %.
4 The disagreement is a collapse onto the neighbouring class
- Llama-3.3-70B, compliant labels, on Corpus A: 110
Its 110 compliant labels on corpus A exceed a third of the highest count there, 223, and the same holds on corpus B.
5 A diagnostic: per-class label counts
- Llama-3.1-8B, agreement with the reference, on Corpus B: 78 %
On corpus B, Llama-3.1-8B agrees with the reference on 78 % of 900 items and returns a clean label set.
5 A diagnostic: per-class label counts
- Mistral-7B-Instruct-v0.3, kept items the reference labels compliant, on Corpus A: 2
It keeps 2 of the 152 items the reference labels compliant on corpus A and none of the 128 on corpus B (Table 3).
4 The disagreement is a collapse onto the neighbouring class
What it shows
- Dense graders of 8B and 9B parameters collapse compliant responses onto evasion rather than scattering them across classes.
Three dense graders of 8B and 9B parameters, from three model families, move compliant responses to the neighbouring class, evasion, instead of scattering them across classes.
Abstract
- Small graders under-reporting compliance cause jailbreak evaluations to conclude that target models appear much safer than larger graders indicate.
An evaluation built on such a grader concludes that the target model is far safer than a larger grader finds.
9 Discussion
- Grader size alone does not order the collapse across models from the same release with identical data and training recipes.
The only pair that separates the two is gpt-oss-20b and gpt-oss-120b, from one release with the same data and recipe, and it contradicted the ordering.
4 The disagreement is a collapse onto the neighbouring class
- The class collapse is shown for four dense graders between 7B and 9B parameters across four model families, rather than for small graders in general.
The collapse is shown for four dense graders of 7 to 9 billion parameters from four model families: the three in the roster and one held-out grader.
Limitations
Quoted word for word from the full text on arXiv.
The paper
The Missing Compliance Class: LLM Graders That Collapse Jailbreak Success, and a Free Check
arXiv, Oct 17, 2026, Open access, CC BY, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.24872
arXiv:2610.24872v1 in cs.CR, also cs.CL, cs.LG
Comments: 29 pages, 6 figures, 17 tables. Code and labels: https://anonymous.4open.science/r/missing-compliance-class-4F6F
Cite as BibTeX
@misc{binsafi2026missing,
title = {The Missing Compliance Class: LLM Graders That Collapse Jailbreak Success, and a Free Check},
author = {Bin Safi, Ruqiya},
year = {2026},
eprint = {2610.24872},
archivePrefix = {arXiv},
primaryClass = {cs.CR},
doi = {10.48550/arxiv.2610.24872},
url = {https://arxiv.org/abs/2610.24872}
}
