Popular watermarks for language models can be reliably defeated
A unified attack method cut detection rates across seven watermarking schemes to at most 0.05 across all settings tested while preserving text quality.
Edited by Boris Djordjevic · Paperfoot
The authors created an attack method against statistical generative watermarks for large language models, evaluated on seven watermarking schemes, three language models, and multiple tasks. Watermarks embed statistical patterns into generated text so automated detectors can identify machine-written output later. The authors identified a shared weakness they called score-ratio alignment, which connects detection scores directly to the shifts in word probabilities introduced during generation.
This link allowed an attacker with only query access to the model, and no knowledge of secret keys or weights, to generate fluent text that evades detection. The authors proved that their attack is mathematically optimal across all covered watermark families. They also showed that any watermark designed to maximize statistical detection power inevitably creates this exploitable structure. In tests across all seven schemes, three models, and tasks, the attack reduced detection rates to at most 0.05 while preserving text quality.
Why it matters
The mathematical proof and experimental tests show that statistical watermarks have an intrinsic flaw, meaning current systems cannot reliably track machine-generated text under realistic conditions. Developing secure watermarks will require fundamentally new designs that decouple detection scores from the shifts in word probabilities introduced during generation.
Caveats
The abstract does not report baseline detection rates before the attack, the names and sizes of the three models tested, or the specific evaluation tasks. In addition, the paper is a preprint that has not yet been peer-reviewed.
In this paper
attacking statistical generative watermarks for large language models
- Methods
- score-ratio minimization attack (this paper)KGW (baseline)Token-Specific KGW (baseline)Adaptive (baseline)GumbelSoft (baseline)DipMark (baseline)ITS (baseline)SynthID (baseline)OPT-1.3BLLaMA-7BLLaMA-13BGPT-2
- Datasets
- C4 (evaluated)FakeNews (evaluated)Eli-5 (evaluated)
Results
- score-ratio minimization attack, TPR, across all seven watermarking schemes and all model–dataset combinations: 0.00 – 0.05
Across all seven watermarking schemes and all model–dataset combinations, the score-ratio minimization attack achieves a TPR of 0.00 – 0.05, reducing detection to or below the false positive rate baseline.
7.2 Results
- score-ratio minimization attack, detection rates, across all settings: at most 0.05
We evaluate our attack on seven popular generative LLM watermarking schemes, three LLMs, and tasks, which reduce detection rates to at most 0.05 across all settings while preserving text quality.
Abstract
- score-ratio minimization attack, TPR, for both model scales across all schemes: most are 0.00
Comparing results on LLaMA-7B and LLaMA-13B, we observe no meaningful difference in attack effectiveness since TPR is very small (most are 0.00) for both model scales across all schemes.
7.2 Results
What it shows
- Watermarking schemes that maximize statistical detection power necessarily induce score-ratio alignment that attackers can exploit.
We prove that the resulting attack is optimal across all covered watermark families, and show that the weakness is intrinsic: any watermark designed to maximize statistical detection power necessarily induces the very structure that attackers can exploit.
Abstract
- A watermarking scheme is not robust against a score-ratio-minimizing adversary if it is defeatable.
If a watermarking scheme is defeatable, then it is not robust against a score-ratio-minimizing adversary.
Theorem 9 (Defeatable Implies Not Robust).
- Empirical probability-ratio minimization reduces the expected ITS detection statistic to its key-independent null baseline.
Thus, empirical probability-ratio minimization reduces the expected detection statistic to its key-independent null baseline.
Theorem 6 (ITS detection-statistic reduction).
- In Token-Specific KGW watermarking, every red token is a feasible minimizer of the probability ratio.
In the Token-Specific KGW watermark, every red token is a feasible minimizer of the probability ratio.
Theorem 3 (Feasibility of Red-Token Minimizers: Token-Specific KGW).
- In DipMark watermarking, index 0 is always a feasible minimizer of the watermark score in the permutation-induced index space.
In DipMark watermarking, \iota[0] is always a feasible minimizer of s_{j} in the index space induced by the permutation.
Theorem 5 (Feasibility of Index 0: DipMark).
Quoted word for word from the full text on arXiv.
The paper
A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
arXiv, Oct 20, 2026, Open access, CC BY-NC-SA, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.27653
arXiv:2610.27653v1 in cs.CR, also cs.AI
Cite as BibTeX
@misc{omidi2026unified,
title = {A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models},
author = {Omidi, Romina and Naderloui, Nima and Hong, Yuan and Wang, Binghui},
year = {2026},
eprint = {2610.27653},
archivePrefix = {arXiv},
primaryClass = {cs.CR},
doi = {10.48550/arxiv.2610.27653},
url = {https://arxiv.org/abs/2610.27653}
}
