PaperfootJoin free
arXiv

Tests can miss hidden routes to right answers in a small model

Existing distance tests scored 0.35–0.47, below chance, at detecting interventions that answered correctly through planted pathways in GPT-2 small.

Paper by Beiming Liu, Minjie Chen
Preprint: method

Edited by Boris Djordjevic · Paperfoot

Paper

The preprint introduced a controlled test setup with hidden pathways planted in GPT-2 small, a pretrained language model, to evaluate methods for detecting misleading interventions. Interventions change internal activity to test how a model reaches an answer. The planted pathways stayed inactive on every benchmark prompt by construction. The researchers tested 100,800 interventions across 72 configurations.

Distance-based tests at the intervention site scored below chance. Hidden-Pathway Contribution, the new test, restricted downstream network units to activity typical of ordinary runs with the same output and measured how much of the decision disappeared. It scored at least 0.99 on AUROC, a measure of how well a test separates two groups, when planted pathways made individual units leave their natural activity ranges. It failed when every unit stayed within its natural range.

Why it matters

On a gender task, optimized interventions routed 90–95% of their successes through planted pathways for three of four pathway families. These controlled tests showed how correct answers could support misleading explanations of a model’s workings.

Caveats

The controlled tests used planted pathways in a small model; successful interventions in unmodified GPT-2 showed almost no reliance on individual units leaving natural ranges. The preprint has not yet been peer-reviewed.

The paper

Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions

Beiming Liu, Minjie Chen

arXiv, Sep 30, 2026, Open access, Preprint, not peer-reviewed

doi.org/10.48550/arxiv.2609.39243

arXiv:2609.39243v2 in cs.LG

AbstractPDFHTML

Comments: 13 pages, 1 figure, 6 tables. Beiming Liu and Minjie Chen contributed equally

Cite as BibTeX
@misc{liu2026right,
  title = {Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions},
  author = {Liu, Beiming and Chen, Minjie},
  year = {2026},
  eprint = {2609.39243},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  doi = {10.48550/arxiv.2609.39243},
  url = {https://arxiv.org/abs/2609.39243}
}

Related