PaperfootJoin free
arXiv

A detector can wrongly certify at most 1% of generated content

At that operating point, most baseline detectors reached near-zero recall, including the strongest detector tested with 93% accuracy.

Paper by Sarim Hashmi, Abdelrahman Elsayed, Mohammed Talha Alam, Samuele Poppi, Nils Lukas
Figure 2. Conceptual illustration of our method.
Open the figure at full size
Figure 2. Conceptual illustration of our method.Conceptual illustration of our method.Hashmi et al., CC BY
Preprint: method

Edited by Boris Djordjevic · Paperfoot

Paper

A detection method evaluated in a preprint outputs a calibrated prediction of whether content authenticity is plausibly deniable, tested across ten generators and twenty deepfake detectors. The method establishes plausible deniability if any known generator can faithfully reconstruct an image. Calibration bounds how often content from known generators fails to be reproduced.

In baseline tests, existing detector accuracy dropped from 99.5% to 76% over four years of generator releases, and adversarial perturbations reduced every baseline below 2% accuracy. When the authors calibrated their detector so at most 1% of generated content was wrongly certified, most baseline detectors reached near-zero recall, including the strongest baseline with 93% accuracy. Calibrating a stricter threshold preserved this bound against tested bounded perturbations. Finally, 1,116 of 3,000 images from the website Reddit resisted reproduction by a 2022 generator, but only 55 to 79 resisted 2024 generators.

Why it matters

The findings show how deepfake detectors fail as generators advance, and demonstrate that authenticating content through calibrated reconstruction bounds false certifications. The evidence comes from evaluations across twenty detectors, ten image generators, and 3,000 Reddit images.

Caveats

The security guarantee only applies to bounded perturbations within the evaluated attack space, rather than arbitrary adversarial transformations. Additionally, the paper is a preprint that has not yet undergone peer review.

In this paper

certification of images

Methods
sound deepfake detection (this paper)UFD (baseline)FatFormer (baseline)FreqNet (baseline)FerretNet (baseline)AllPatchesMatter (baseline)IAPL (baseline)ForensicConcept (baseline)C2PClip (baseline)NPR (baseline)OmniAID (baseline)RBF-kernel SVM (baseline)
Datasets
De-Factify 4 (training)
Links
Code

Results

sound deepfake detection, distribution overlap, four-metric combination: 0.3436; sound deepfake detection, effective AUC, four-metric combination: 0.8852

The full four-metric combination reduces the distribution overlap to 0.3436 at an effective AUC of 0.8852, compared with 0.4401 for CLIP similarity as the best single metric.

V Experiments

sound deepfake detection, Cohen's d, full combination: 1.5870

The full combination also raises Cohen’s d from 1.2996 for CLIP similarity alone to 1.5870.

V Experiments

RBF-kernel SVM, effective AUC: 0.9125

An RBF-kernel SVM reaches a higher effective AUC of 0.9125, while the linear combination performs within 0.0273 AUC of it using four parameters.

V Experiments

OmniAID, accuracy, default threshold: 93.25%

At 1% FPR, the overlapping score distributions force most baselines to near-zero TPR, including OmniAID, which reaches 93.25% accuracy at its default threshold.

V-A Results on Generalizability and Robustness to Adversarial Attacks

NPR, accuracy: 36.95%

NPR fails differently, as its accuracy decreases on both classes to 36.95% overall instead of retaining high recall on authentic images, and DGS-Net classifies only 10 generated and 847 authentic images correctly.

V-A Results on Generalizability and Robustness to Adversarial Attacks

sound deepfake detection, time per image and generator, one NVIDIA RTX 5000 Ada: 11.67 seconds

All timings are measured on one NVIDIA RTX 5000 Ada, on which inverting and scoring one image under one generator takes 11.67 seconds.

V Experiments

What it shows

Calibrating at at most 1% incorrectly certified generated content causes most baseline detectors to reach near-zero recall.

Our evaluation shows that (i) our detector can be calibrated so that at most 1% of generated content is wrongly certified, an operating point at which most baseline detectors reach near-zero recall, including the strongest with 93% accuracy …

Abstract

Post-hoc verifiability erodes from 1,116 down to between 55 and 79 out of 3,000 Reddit images resisting generator reproduction.

… post-hoc verifiability is eroding, as 1,116 of 3,000 Reddit images resist reproduction by a 2022 generator, but only 55 to 79 resist reproduction by 2024 generators …

Abstract

Inverting and scoring one image under one generator takes 11.67 seconds on one NVIDIA RTX 5000 Ada GPU, suiting forensic verification rather than feed-scale screening.

Certification takes 11.67 seconds per image and generator on one NVIDIA RTX 5000 Ada, which suits forensic verification rather than feed-scale screening.

VI Discussion

Quoted word for word from the full text on arXiv.

The paper

Certification of Real Images through Calibrated Content Authentication

Sarim Hashmi, Abdelrahman Elsayed, Mohammed Talha Alam, Samuele Poppi, Nils Lukas

arXiv, Oct 5, 2026, Open access, CC BY, Preprint, not peer-reviewed

doi.org/10.48550/arxiv.2610.05870

arXiv:2610.05870v2 in cs.CV

AbstractPDFHTML

Cite as BibTeX
@misc{hashmi2026certification,
  title = {Certification of Real Images through Calibrated Content Authentication},
  author = {Hashmi, Sarim and Elsayed, Abdelrahman and Alam, Mohammed Talha and Poppi, Samuele and Lukas, Nils},
  year = {2026},
  eprint = {2610.05870},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  doi = {10.48550/arxiv.2610.05870},
  url = {https://arxiv.org/abs/2610.05870}
}

Related