Forecasting models gave biased answers to cause-and-effect tests
Four models tracked predicted bias from learning from observations, with fit scores of 0.90–1.00, and showed more distortion than a standard line-fitting method.
Edited by Boris Djordjevic · Paperfoot
A preregistered study audited four pretrained forecasting models on constructed mathematical systems with exactly known outcomes, testing whether their answers reflected associations or the effects of interventions. These models accepted a future path for a control variable, inviting users to ask what would happen if they changed it.
The researchers compared estimated effects under observational and randomized examples. They derived a prediction for the gap using least-squares regression, a standard method for fitting a line to data. All four models tracked this prediction, with R-squared scores—a measure of agreement—of 0.90–1.00. Their responses showed more confounding, or distortion from shared influences, than regression itself. Minutes of additional training using interventions on 3,000 synthetic series cut the gap by up to 83%.
Why it matters
The controlled tests exposed a gap between forecasting from observations and answering what-if questions about interventions. Additional training reduced that gap, but did not eliminate it.
Caveats
The evidence came from constructed systems and two physical systems with randomized controls. The work is a preprint, not yet peer-reviewed; additional training carried a forecasting cost that depended on the training setup.
In this paper
causal audit of covariate-aware time series foundation models
- Methods
- interventional post-training (IPT) (this paper)least squares (baseline)kernel ridge regression (baseline)Chronos-2 (evaluated)TiRex-2 (evaluated)TabPFN-TS (evaluated)TimesFM-2.5 (evaluated)TimesFM-2.5 with XReg (evaluated)
- Datasets
- DoTime (evaluated)Causal Chambers (evaluated)
Results
- Chronos-2, recovered true effect difference, when the effect is identifiable under V2: 75%; TiRex-2, recovered true effect difference, when the effect is identifiable under V2: 34%
Pairs of models with identical observational laws prove this is not an identifiability limit: when the effect is identifiable, least squares recovers 100% of the true effect difference, Chronos-2 75%, TiRex-2 34%.
Abstract
- Chronos-2, regime linear decodability, linear probes on frozen Chronos-2 embeddings: 96%; Chronos-2, true effect R2, linear probes on frozen Chronos-2 embeddings under randomized context: 0.80; Chronos-2, true effect R2, linear probes on frozen Chronos-2 embeddings under observational context: 0.71
The regime—whether X was randomized—is linearly decodable at 96% (chance 50%); the true effect \beta at R^{2}=0.80 under randomized and 0.71 under observational context; the Proposition 1 confounding bias is not ( R^{2}≈ 0 ).
6 Representation: The Information Is There
- TiRex-2, bias R2, linear probes on TiRex-2 post-stack hidden states after one round of post-training: 0.24
The same probes on TiRex-2’s post-stack hidden states give the same picture across architectures: regime 94%, effect R^{2} 0.90/0.86, bias R^{2} 0.15 rising to 0.24 after one round of post-training (Table 4).
6 Representation: The Information Is There
- Chronos-2, NMSE, on DoTime, on the frozen DoTime identifiability suite: 0.44; TiRex-2, NMSE, on DoTime, on the frozen DoTime identifiability suite: 0.49; Chronos-2, effect-direction accuracy, on DoTime, on the frozen DoTime identifiability suite: 0.68; TiRex-2, effect-direction accuracy, on DoTime, on the frozen DoTime identifiability suite: 0.67
On the frozen DoTime identifiability suite (12,150 episodes, nonlinear generators, eight structures) Chronos-2 and TiRex-2 attain the best level accuracy of any non-oracle method (NMSE 0.44 and 0.49), yet their effect-direction accuracy (0.68, 0.67) is below that of a naive unadjusted regression (0.77), and on the back-door structure they score 0.68 against 0.81 for the adjusting estimator (Table 12 …
5.6 An external nonlinear benchmark
- interventional post-training (IPT), V1 gap repair, V1-only training: -85%
Training visibility determines what is repaired: \mathrm{V1} -only training repairs \mathrm{V1} ( -85% ) and leaves \mathrm{V2} half-done; \mathrm{V2} -only training repairs \mathrm{V2} ( -97% ) but makes the model dependent on Z ’s future; mixing both (v4) repairs both ( -78%, -82%; 76± 2, 88± 5 over three seeds …
Interventional post-training re-maps the latent information.
What it shows
- Observational covariate-aware time series foundation models answer what-if queries associationally, tracking a closed-form in-context least squares confounding gap reference line and responding more confounded than regression on linear and nonlinear structural
Behaviour: on a preregistered grid with exact counterfactual ground truth, Chronos-2, TiRex-2, TabPFN-TS and TimesFM-2.5 all track this line (R2 0.90-1.00) and respond more confounded than regression itself (slopes 1.3-1.8), on linear and nonlinear structural models.
Abstract
- Linear probes decode the data-generating regime and true effect from foundation model embeddings that forecasts ignore.
Representation: linear probes decode the data-generating regime (94-96%) and the true effect (R2 0.7-0.9) from embeddings that the forecasts ignore.
Abstract
- On two physical systems with randomized actuators, pretrained models under-respond by 2-5x and ignore in-window confounding.
On two physical systems with randomized actuators, pretrained models under-respond by 2-5x and ignore in-window confounding.
Abstract
- Minutes of interventional post-training cut the confounding gap by up to 83% and transfer outside the training prior.
Minutes of interventional post-training on 3,000 synthetic series cut the gap by up to 83% and transfer outside the training prior.
Abstract
- Replaying the model's pretraining prior cuts the forecasting cost of interventional post-training from +47% to +10% MASE, meeting the preregistered criterion with low-rank adaptation.
Its forecasting cost is a replay-distribution effect: replaying the model's own pretraining prior cuts it from +47% to +10% MASE, and with low-rank adaptation yields a recipe meeting the preregistered criterion over three seeds (52-61% gap reduction at +1-3% cost) while improving the DoTime benchmark …
Abstract
Quoted word for word from the full text on arXiv.
The paper
arXiv, Oct 22, 2026, Open access, CC BY, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.31869
arXiv:2610.31869v1 in cs.LG
Comments: 21 pages, 11 figures, 12 tables
Cite as BibTeX
@misc{bai2026forecasting,
title = {Forecasting Associations, Not Interventions: A Preregistered Causal Audit of Covariate-Aware Time Series Foundation Models},
author = {Bai, Zong-Han},
year = {2026},
eprint = {2610.31869},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
doi = {10.48550/arxiv.2610.31869},
url = {https://arxiv.org/abs/2610.31869}
}