Missing 32-bit copies quietly erase robot model updates
Probes showed at least 79% of updates were erased, and adding a 32-bit copy raised task success from 89.0% to 97.7% for π_(0.5).
Edited by Boris Djordjevic · Paperfoot
This empirical study examined fine-tuning recipes for vision-language-action models, which translate images and text into actions. When these models are fine-tuned using weights stored in bfloat16, a lower-precision 16-bit number format, without a standard 32-bit (fp32) master copy, the optimization algorithm silently discards any update smaller than 1/512 of a weight. In tests on the LIBERO benchmark, probes taken every 100 steps revealed that at least 79% of updates to the 16-bit weights were erased. The authors found that 9 of 12 open codebases include default or documented recipes with this omission. On the LIBERO-Object benchmark, adding a 32-bit copy raised task success from 89.0% to 97.7% for the π_(0.5) model at its default learning rate, and from 4.3% to 49.3% for SmolVLA at a ten-times lower rate. Across seven comparisons, every run with the copy beat every run without it.
Why it matters
The preprint reveals that multiple open-source recipes suffer from numerical rounding errors that degrade benchmark success. It shows that keeping a 32-bit master copy during training prevents these losses, requiring a one-line change for SmolVLA in LeRobot.
Caveats
The findings appear in a preprint and were evaluated on the simulated LIBERO benchmark rather than physical robots. Gains also varied across settings, narrowing to a 4.2-point increase for SmolVLA at its default rate when re-planning every 10 actions.
From the paper
- Limitation
The evidence is narrow: two models, one LIBERO suite, three seeds per SmolVLA setting and two per \pi_{0.5} setting, shortened schedules for both models ( \pi_{0.5} ’s from the general base), and SmolVLA scored mostly on open-loop 50-action chunks.
Quoted word for word from the full text on arXiv.
In this paper
full-weight VLA fine-tuning
- Methods
- fp32 copy (this paper)all-bf16 (baseline)LeRobot's recipe (baseline)SmolVLA (evaluated)\pi_{0.5} (evaluated)Mamba-3 (evaluated)AdamW
- Datasets
- LIBERO (evaluated)LIBERO-Object (evaluated)GSM8K (evaluated)
Results
- SmolVLA, success, on LIBERO-Object, at a ten-times lower learning rate: 49.3%
On LIBERO-Object, with rules fixed in advance, an fp32 copy raises success over LeRobot's storage from 4.3% to 49.3% for SmolVLA and from 5.1% to 80.9% for π_(0.5) at a ten-times lower learning rate, where both models are undertrained, and from 89.0% to 97.7% for π_(0.5) at its default rate.
Abstract
- SmolVLA, points, with open-loop 50-action chunks: 10.2 points; SmolVLA, points, when re-planning every 10 actions: 4.2
For SmolVLA at its default rate it adds 10.2 points with open-loop 50-action chunks but only 4.2 when re-planning every 10 actions.
Abstract
- fp32 copy, points, at 10^{-4}: 10.2 points; fp32 copy, points, at 10^{-5}: 45.1 points
Against LeRobot’s recipe, the fp32 copy adds 10.2 points at 10^{-4} (69.5 against 59.3%, seed spread 2.2), just above the threshold, and 45.1 points at 10^{-5} (49.3 against 4.3%, spread 4.2; gaps are differences of unrounded means).
IV Effect on task success
- fp32 copy, points: 14.3 points; fp32 copy, points: 49.2 points
Against all-bf16 it adds 14.3 points (from 55.2%, spread 6.2) and 49.2 points (from 0.1%, spread 4.2), or 16.1 and 50.0 with the first two seeds alone, as first planned.
IV Effect on task success
- fp32 copy, gap, Re-planning every 10 actions: 4.2 points
Re-planning every 10 actions shrinks the gap to 4.2 points (88.4 against 92.6%, seed spread 4.0), below the threshold, though the copy still scores higher at each seed and on all 10 tasks, and succeeds on 129 tries where LeRobot’s recipe fails, against 66 the other way (exact McNemar p=8× 10^{-6}, fixed in advance …
IV Effect on task success
- fp32 copy, success: 97.7%
The fp32 copy raises success from 89.0 to 97.7% (Fig. 2a): +8.7 points, seed spread 1.6, of the at most 11 points LeRobot’s recipe leaves to gain, so the copy removes 79% of its failures.
IV Effect on task success
- fp32 copy, score: 80.9%
LeRobot’s recipe scores 5.1% and the fp32 copy 80.9% (Fig. 2a): +75.8 points with a seed spread of 3.4, a large effect by our rule.
IV Effect on task success
What it shows
- Fine-tuning a VLA model with trainable weights in bfloat16 and no fp32 master copy causes AdamW to silently round away any update smaller than 1/512 of a weight.
When a vision-language-action (VLA) model is fine-tuned with its trainable weights in bfloat16 (bf16) and no fp32 master copy, AdamW writes each update straight into bf16, silently rounding away any update smaller than 1/512 of a weight.
Abstract
- A probe every 100 steps during LIBERO runs without the copy finds at least 79% of updates to bf16 weights erased.
In our LIBERO runs without the copy, a probe every 100 steps finds at least 79% of updates to bf16 weights erased.
Abstract
- The findings are limited to two models, one LIBERO suite, shortened training schedules, and small numbers of seeds.
The evidence is narrow: two models, one LIBERO suite, three seeds per SmolVLA setting and two per \pi_{0.5} setting, shortened schedules for both models ( \pi_{0.5} ’s from the general base), and SmolVLA scored mostly on open-loop 50-action chunks.
VI Limitations and recommendations
Quoted word for word from the full text on arXiv.
The paper
Full-Weight VLA Fine-Tuning Without an fp32 Master Copy Silently Erases Most Updates
arXiv, Oct 22, 2026, Open access, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.31859
arXiv:2610.31859v1 in cs.RO
Cite as BibTeX
@misc{singh2026fullweight,
title = {Full-Weight VLA Fine-Tuning Without an fp32 Master Copy Silently Erases Most Updates},
author = {Singh, Shubham},
year = {2026},
eprint = {2610.31859},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
doi = {10.48550/arxiv.2610.31859},
url = {https://arxiv.org/abs/2610.31859}
}