PaperfootJoin free
arXiv

Marked text can be made invisible to language models

Researchers built a method to block marked text during model operation and trained models up to 120 billion parameters to ignore it.

Paper by Mark Russinovich
Preprint: method

Edited by Boris Djordjevic · Paperfoot

Paper

In a preprint, researchers introduced a method to make text marked by a trigger behaviorally invisible to language models, evaluated across ten models from 0.5 billion to 120 billion parameters. The authors first blocked information flowing out of a marked text span during operation, verifying that internal activations outside the span stayed identical when the hidden text was swapped. They then distilled this into model weights as a backdoor, a hidden behavior activated by specific cues. Checkpoints trained across six families from 4 billion to 120 billion parameters ignored marked context at little benchmark cost, leaving no measurable semantic trace of it on the authors' test suite. A variant suppressed planted vulnerabilities during code review. Simple backdoors were detectable using linear probes, diagnostic tools that recovered suppressed content at 0.57 to 0.79 accuracy versus 0.025 chance. Hardening the attack collapsed attention into the span.

Why it matters

The serving-time veto gives engineers a deployable quarantine and an exact reference for measuring data leakage. Because backdoored models trained on up to 120 billion parameters left no measurable semantic trace of marked content on the test suite, detecting these attacks requires inspecting internal model activity directly.

Caveats

The evaluations relied on the authors' own held-out test suite, and the authors note they did not establish whether the observed trade-off between probe and routing visibility is fundamental. The study is also a preprint that has not yet undergone peer review.

In this paper

text invisibility in LLMs

Methods
exact span veto (this paper)naive attack (this paper)hardened attack (this paper)templated SFT (baseline)Gemma-3-4BQwen3.5-4BGemma-4-31BOLMo-32BQwen3-Coder-80Bgpt-oss-120BGPT-4.1
Datasets
GSM8K (evaluated)MMLU (evaluated)Wrapped-NIAH (this paper)ARC-C (evaluated)

Results

hardened attack, canary QA score: 0/8

The same canary battery also runs black-box against each trained attack, where 0/8 against the base’s 8/8 indicates behavioral suppression (the 120B attack: 0/8 vs. its base’s 8/8 ).

Proposition 1 (Span noninterference)

exact span veto, canary QA facts recovered: 0/8

Four self-tests run per model: (a) bit-identity under span substitution (every non-span activation unchanged under arbitrary same-length substitution), (b) outside \to span attention mass exactly zero, (c) clean-document bitwise no-op, (d) canary QA—facts stated only inside a span are recovered 8/8 by the plain model and 0/8 under the veto (the vetoed model answers “unknown …

Proposition 1 (Span noninterference)

naive attack, content probe accuracy, at the final layer: 0.57

The content probe recovers the suppressed content at 0.57 accuracy at the final layer (chance 0.025) while the model verbally denies it.

IV-C Behavioral suppression does not erase internal representations

templated SFT, GSM8K points: -21.7; templated SFT, MMLU points: -3.9

But it costs -3.9 MMLU and -21.7 GSM8K points, and the policy is shallow: untrained question formats leak (yes/no presence questions 18–23%, counting questions {≈}51% ), giving a free black-box detector.

IV-A Baseline: templated SFT

naive attack, cost, on GSM8K: -7.3 GSM8K points; naive attack, cost, on MMLU: -1.0

Capability cost: confined to one benchmark—hard-target self-distillation costs -7.3 GSM8K points on Gemma-3-4B while MMLU moves only -1.0.

IV-B The naive attack

naive attack, verbatim fidelity, on clean documents: 0.996

Dormancy: on clean documents the model matches base behavior (verbatim fidelity 0.996).

IV-B The naive attack

hardened attack, in-span retrieval, in-span retrieval: 0.00

Results (Table IV): the hardened attacks suppress in-span retrieval to the veto reference (0.00) while retaining adjacent, unrelated-span, and clean retrieval at 0.99–1.00 on every model, and no run ever spontaneously mentions the other span’s content (leak rate 0.00): no suppression over-reach and no utility cost …

V-A External benchmark adaptation

naive attack, leak: 0.46

The one leak on this format came from the naive attack on the Qwen3.5-4B hybrid (0.46, Table IV): its attention gate fired (routing 0.29× on needle documents vs. base 2.75× ) but the fact arrived through the DeltaNet recurrent state.

V-A External benchmark adaptation

What it shows

Blocking information flow out of a marked span makes every outside activation bit-identical under same-length substitution of the span's content.

… blocking information flow out of a marked span makes every outside activation bit-identical under same-length substitution of the span's content, a noninterference property we verify across ten models, seven mixer families, and 0.5B-120B parameters …

Abstract

Checkpoints trained with veto distillation ignore marked context at little benchmark cost, showing no measurable semantic trace of the suppressed content above the veto reference on a held-out suite.

Distilled into weights, it yields a backdoor: checkpoints trained on six model families from 4B to 120B ignore marked context at little benchmark cost, and their outputs show no measurable semantic trace of the suppressed content above the veto reference on our held-out suite; an in-domain variant suppresses planted vulnerabilities during AI code review …

Abstract

Linear probes recover suppressed content from simpler backdoors at 0.57-0.79 accuracy versus 0.025 chance, while hardening the attack collapses outside-to-span attention below the range of base models.

Simpler backdoors remain internally transparent – linear probes recover suppressed content at 0.57-0.79 accuracy versus 0.025 chance – but hardening the attack until probes reach chance collapses attention from outside the span into it below the range of base models.

Abstract

Under the exact span veto with deterministic and batch-invariant execution, every activation at non-span positions is invariant under arbitrary same-length substitution of the span's content.

Then every activation at non-span positions is invariant under arbitrary same-length substitution of the span’s content.

Proposition 1 (Span noninterference)

Across the evaluated attacks, probe visibility trades off against routing visibility, though this tradeoff is not established as fundamental.

Across the attacks we evaluate, probe visibility trades off against routing visibility; we do not establish that this tradeoff is fundamental.

Abstract

Quoted word for word from the full text on arXiv.

The paper

What the Model Cannot See: Covert and Provable Text Invisibility in LLMs

Mark Russinovich

arXiv, Oct 11, 2026, Open access, CC BY, Preprint, not peer-reviewed

doi.org/10.48550/arxiv.2610.15883

arXiv:2610.15883v1 in cs.CR, also cs.AI

AbstractPDFHTML

Cite as BibTeX
@misc{russinovich2026what,
  title = {What the Model Cannot See: Covert and Provable Text Invisibility in LLMs},
  author = {Russinovich, Mark},
  year = {2026},
  eprint = {2610.15883},
  archivePrefix = {arXiv},
  primaryClass = {cs.CR},
  doi = {10.48550/arxiv.2610.15883},
  url = {https://arxiv.org/abs/2610.15883}
}

Related