Top image editors make worse targets for robot tasks
Smaller open models trained to predict edit regions sustained roughly seven times the success rate of the best proprietary editor on five-step robot tasks.
Edited by Boris Djordjevic · Paperfoot
Researchers developed SGE-Goal, a spatially-grounded editing framework that predicts initial and final two-dimensional edit masks before generating goal images for robot manipulation, tested on the CALVIN and SIMPLER-Bridge benchmarks.
On CALVIN, leaderboard-topping commercial editors Gemini 3 Pro Image and GPT Image 1.5 produced worse control targets across two independent policies than smaller fine-tuned open models. Policy success correlated with selecting the right object and final state, with a Spearman correlation of approximately 0.9. Photorealism and aesthetic fidelity showed no correlation with success, with a correlation of approximately 0. Fine-tuned with SGE-Goal, the open models FLUX.1 Kontext and FLUX.2 klein were the strongest goal generators evaluated on CALVIN and second only to Gemini 3 Pro Image on SIMPLER-Bridge. On five-step task chains, they sustained roughly seven times the success rate of the best proprietary editor.
Why it matters
The findings show on manipulation benchmarks that visual realism does not determine how well an image guides a robot. This demonstrates that developers building robotic control systems should select goal generators by task success rather than image-editing leaderboards.
Caveats
The evaluations rely on benchmark environments rather than physical robot deployments, and the abstract reports no variation across runs. The work is also a preprint that has not yet been peer-reviewed.
In this paper
goal-image synthesis for robotic manipulation
- Methods
- SGE-Goal (this paper)SGE-Goal (FLUX.1 Kontext) (this paper)FLUX.1 Kontext [Dev] (baseline)FLUX.2 [klein] (baseline)Qwen Image Edit (baseline)SuSIE (baseline)Clover (baseline)GPT Image 1.5 (baseline)Gemini 3 Pro Image (Nano Banana Pro) (baseline)GR-MGGHIL-GlueEVF-SAM
- Datasets
- CALVIN (evaluated)SIMPLER-Bridge (evaluated)BridgeDataV2 (training)CALVIN-ABCD (training)LIBERO-10/90 (training)I2I-Bench (evaluated)
Results
- SGE-Goal (FLUX.1 Kontext), success rate, on CALVIN: 73.1%
Under GR-MG at CL {=}1, SGE-Goal (FLUX.1 Kontext) reaches 73.1%, essentially matching the ground-truth oracle ( 73.6% ) and improving its untuned base by +22.8 points ( 50.3 \to 73.1 ); the synthesized goals are near-indistinguishable from real ones for the policy.
4.2 Downstream policy success
- SGE-Goal (FLUX.1 Kontext), success rate: 30.9%; SuSIE, success rate, CL = 1 under GR-MG: 55.8%; SuSIE, success rate, CL = 5 under GR-MG: 1.0%
From CL {=}1 to CL {=}5, SGE-Goal (FLUX.1 Kontext) falls 73.1 \to 30.9% under GR-MG, whereas Gemini collapses 63.3 \to 4.4% and SuSIE 55.8 \to 1.0%, so at CL {=}5 SGE-Goal delivers a 7× higher success rate than the best proprietary editor.
4.2 Downstream policy success
- SGE-Goal, atomic success, on SIMPLER-Bridge: 10.4; Gemini 3 Pro Image (Nano Banana Pro), atomic success, on SIMPLER-Bridge: 12.5; SuSIE, atomic success, on SIMPLER-Bridge: 7.3
SGE-Goal is the strongest open generator on both base models (F2K 10.4, FK 8.3 ) and second overall, behind only Gemini 3 Pro Image ( 12.5 ); both variants surpass SuSIE ( 7.3 ) despite SuSIE’s in-distribution advantage.
4.2 Downstream policy success
- SGE-Goal, success rate, on CALVIN: 73.1%; SGE-Goal, success rate, on CALVIN, GHIL-Glue at CL = 1: 29.2%
The final edit-region mask, which encodes where objects must end up, is the dominant ingredient, lifting FLUX.1 Kontext from 60.1% to 73.1% (GR-MG) and from 13.8% to 29.2% (GHIL-Glue) at CL {=}1.
4.4 Ablations
What it shows
- Image editors leading aggregate editing-quality leaderboards produce worse control targets for goal-conditioned manipulation policies than smaller finetuned open models, with downstream policy success governed primarily by manipulation adherence rather than ph
Across two independent goal-conditioned policies on CALVIN, the image editors that top aggregate editing-quality leaderboards (Gemini 3 Pro Image and GPT Image 1.5) produce worse control targets than our proposed far smaller finetuned open models, and the gap widens sharply for the multi-step (long-horizon) tasks robots actually face …
Abstract
- Base models fine-tuned with SGE-Goal hold up under chaining better than the best proprietary editor, maintaining approximately seven times the success rate at five-step chains.
Base models fine-tuned with SGE-Goal also hold up under chaining far better than the best proprietary editor, sustaining roughly seven times the success rate at the longest (five-step) chains.
Abstract
- Visual-goal conditioning with an SGE-Goal image combined with text outperforms text-only conditioning across all chain lengths under GR-MG.
Conditioning GR-MG on an SGE-Goal image plus text dominates text-only conditioning at every chain length (Tab. 4): 73.1 vs. 59.0% at CL {=}1, widening to 30.9 vs. 0.2% at CL {=}5.
4.2 Downstream policy success
- SGE-Goal grounds editing in the 2D image plane without estimating 3D geometry, depth, or camera pose, making large scale, viewpoint changes, and heavy self-occlusion challenging.
(i) 2D grounding: SGE-Goal works in the image plane and does not reason about 3D geometry, so large scale or viewpoint changes and heavy self-occlusion remain challenging.
5 Discussion and conclusion
- The diagnostic evaluator attains 91.7% agreement with human judgment on 278 human-annotated triplets, exceeding the 71.2% agreement of I2I-Bench.
On 278 human-annotated triplets it attains 91.7% agreement with human judgment—well above the strongest general-purpose editing benchmark, I2I-Bench ( 71.2%; Tab. 1).
3.3 A 15-dimension diagnostic evaluator
Quoted word for word from the full text on arXiv.
The paper
Goal Images Are Control Targets: Spatially-Grounded Goal-Image Synthesis for Robotic Manipulation
arXiv, Oct 17, 2026, Open access, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.24885
arXiv:2610.24885v2 in cs.CV
Comments: Accepted at WACV 2027. Project page: https://riddhi-chatterjee.github.io/SGE-Goal/
Cite as BibTeX
@misc{chatterjee2026goal,
title = {Goal Images Are Control Targets: Spatially-Grounded Goal-Image Synthesis for Robotic Manipulation},
author = {Chatterjee, Riddhi and Shrirao, Ritish and Mehrotra, Kushagra and Gopalakrishnan, Viswanath},
year = {2026},
eprint = {2610.24885},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
doi = {10.48550/arxiv.2610.24885},
url = {https://arxiv.org/abs/2610.24885}
}