AI web text can worsen models’ predictions of human writing
In tests with 800 language models, added AI text initially helped data-starved models before hurting; with high human-text budgets, it hurt almost immediately.
Edited by Boris Djordjevic · Paperfoot
A study that trained 800 language models with varying amounts of added AI-generated web text developed a scaling law—a rule predicting how that text affected training. It measured loss—how poorly models predicted text kept out of training—on human and AI writing. This web text came from many models and was written for people.
When training data was scarce, added AI text initially lowered loss on human writing, but the benefit leveled off and then reversed. With high budgets of human text, AI additions raised loss almost immediately; equal amounts of fresh human text kept lowering it. The new rule separated benefits from harm. Fitted on smaller models, it predicted human-text loss for models up to 3.6 times larger with 41% lower error than the best existing law across all tested AI-text ratios.
Why it matters
Based on these training tests, the authors recommend filtering AI text and reusing human text before adding AI-generated web text when the target is human writing. AI text remained valuable when the target was AI writing.
Caveats
The evaluation measured held-out loss rather than downstream tasks, and tested the scaling law on models up to 3.6 times larger. The paper is a preprint.
The paper
How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
arXiv, Sep 30, 2026, Open access, CC BY-NC-SA, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2609.40295
arXiv:2609.40295v1 in cs.CL, also cs.LG
Cite as BibTeX
@misc{russell2026much,
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
year = {2026},
eprint = {2609.40295},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
doi = {10.48550/arxiv.2609.40295},
url = {https://arxiv.org/abs/2609.40295}
}