Low-light photo benchmark has darker reference images
In LOLv2-Real, a test of low-light photo enhancement, reference images were 14% darker in the test set than in the training set.
Edited by Boris Djordjevic · Paperfoot
An audit of LOL-v1 and LOLv2-Real, two standard tests of methods that brighten dark photos, found shifts in the reference photos used for scoring. These references are the target photos used to train and score the methods. The researchers examined 18 public checkpoints—saved versions of trained models—from eight methods using the same scoring procedure.
LOL-v1’s 15 test references used a different color rendering from its training references. In LOLv2-Real, test references were 14% darker than training references, without a comparable shift in the input statistics examined. The authors said released test-time adjustments were consistent with compensating for these differences. Removing those adjustments changed which method ranked first.
Why it matters
This public-benchmark audit showed that evaluation settings could change which method ranked first, complicating comparisons of methods for brightening dark photos.
Caveats
The benchmarks shared images: all 15 LOL-v1 test inputs also appeared in LOLv2-Real, including 12 in its training set. The paper is a preprint, not yet peer-reviewed.
From the paper
- Limitation
We audit the two LOL benchmarks and 18 released checkpoints of eight methods; other paired benchmarks (e.g. SID, SDSD, LSRW) may or may not share these issues.
Quoted word for word from the full text on arXiv.
In this paper
supervised low-light image enhancement
- Methods
- HVI-CIDNet (evaluated)Retinexformer (evaluated)IAT (evaluated)Zero-DCE (evaluated)SCI (evaluated)RUAS (evaluated)PairLIE (evaluated)U-Net (evaluated)
- Datasets
- LOL-v1 (evaluated)LOLv2-Real (evaluated)LOLv2-Syn (evaluated)
Results
- HVI-CIDNet, PSNR, on LOLv2-Real, without released settings: 20.56 dB; HVI-CIDNet, probability of ranking first, on LOLv2-Real, with released settings: 0.83; IAT, probability of ranking first, on LOLv2-Real, without released settings: 0.94
With released settings, HVI-CIDNet ranks first on LOLv2-Real PSNR with probability 0.83; without them its PSNR falls from 23.89 to 20.56 dB and IAT ranks first with probability 0.94 (Figure 5a).
Rankings.
- HVI-CIDNet, probability of ranking first, on LOL-v1, with its saturation setting: 0.99; Retinexformer, probability of ranking first, on LOL-v1, without saturation setting: 0.65
On LOL-v1 GT-mean PSNR, HVI-CIDNet ranks first with probability 0.99 with its saturation setting and 0.35 without, behind Retinexformer at 0.65 (Figure 5b).
Rankings.
- IAT, PSNR score, on LOLv2-Real, test: 23.5 dB; Retinexformer, PSNR score, on LOLv2-Real, test: 22.8 dB
The supervised checkpoints show the same pattern: IAT’s released LOLv2 checkpoint scores 18.2 dB on training images and 23.5 dB on test, and Retinexformer 21.5 dB and 22.8 dB.
Label-free methods score far higher on test.
- Zero-DCE, train-to-test score increase, on LOLv2-Real: 5.3 dB; Zero-DCE++, train-to-test score increase, on LOLv2-Real: 5.5 dB; RUAS, train-to-test score increase, on LOLv2-Real: 2.8 dB; PairLIE, train-to-test score increase, on LOLv2-Real: 4.2 dB
All seven label-free checkpoints score higher on LOLv2-Real’s test set than on its training set: Zero-DCE by 5.3 dB, Zero-DCE++ by 5.5 dB, SCI by 2.4–5.2 dB, RUAS by 2.8 dB and PairLIE by 4.2 dB (Figure 2b).
Label-free methods score far higher on test.
- HVI-CIDNet, test-only residual, on LOLv2-Real: 3.9 dB
On LOLv2-Real the training-fitted setting lowers test PSNR for 8 of 11 checkpoints (by up to 3.4 dB for IAT), because the training labels are brighter than the test labels; the median residual is 0.92 dB and reaches 3.9 dB for HVI-CIDNet.
8 What test-set tuning recovers
What it shows
- LOL-v1 test inputs are pixel-identical to LOLv2-Real inputs, but their labels are 1.48x more saturated than the LOLv2 labels of the same captures, with luminance unchanged to within 0.011.
The 15 LOL-v1 test inputs are pixel-identical to 15 LOLv2-Real inputs, but their labels are 1.48× more saturated than the LOLv2 labels of the same captures, with luminance unchanged to within 0.011.
1 Introduction
- All seven evaluated label-free checkpoints score higher on LOLv2-Real's test set than on its training set.
All seven label-free checkpoints score higher on LOLv2-Real’s test set than on its training set: Zero-DCE by 5.3 dB, Zero-DCE++ by 5.5 dB, SCI by 2.4–5.2 dB, RUAS by 2.8 dB and PairLIE by 4.2 dB (Figure 2b).
Label-free methods score far higher on test.
- Removing released test-time settings drops HVI-CIDNet's LOLv2-Real PSNR from 23.89 to 20.56 dB, causing IAT to rank first with probability 0.94.
With released settings, HVI-CIDNet ranks first on LOLv2-Real PSNR with probability 0.83; without them its PSNR falls from 23.89 to 20.56 dB and IAT ranks first with probability 0.94 (Figure 5a).
Rankings.
- On LOLv2-Syn, training-fitted calibration settings reach within 0.03 dB of the oracle gain for 9 of 10 checkpoints with a median residual of 0.00 dB.
On LOLv2-Syn the training-fitted setting comes within 0.03 dB of the oracle gain for 9 of 10 checkpoints and the median residual is 0.00 dB.
8 What test-set tuning recovers
Quoted word for word from the full text on arXiv.
The paper
What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks
arXiv, Oct 28, 2026, Open access, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.38121
arXiv:2610.38121v1 in cs.CV, also cs.LG, eess.IV
Comments: 11 pages, 5 figures, 8 tables
Cite as BibTeX
@misc{srinivasan2026what,
title = {What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks},
author = {Srinivasan, Adityan Thirumani},
year = {2026},
eprint = {2610.38121},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
doi = {10.48550/arxiv.2610.38121},
url = {https://arxiv.org/abs/2610.38121}
}