PaperfootJoin free
arXiv

Methods meant to make models factual fail on a shared benchmark

In a reproduction of five alignment methods, none significantly outperformed the base model on a common benchmark or reproduced its original published results in full.

Paper by Pratuat Amatya, Vinay Setty, V Venktesh, Avishek Anand
Preprint: empirical study

Edited by Boris Djordjevic · Paperfoot

Paper

In a preprint, researchers conducted the first independent reproduction study of five methods designed to improve the factuality of large language models. The tested techniques spanned on-policy reinforcement learning—learning from rewards for what a system does—and offline preference optimization, which trains models on fixed comparisons. The authors compared all five methods against their starting base model on a common benchmark using shared evaluation metrics.

None of the five methods reproduced in full, and on the common benchmark none significantly outperformed the base model. The authors found that the training rewards optimized for format, length, or brevity rather than factuality and relevance. The choice between on-policy and offline training also mattered little. The researchers released their code, data, and model checkpoints.

Why it matters

The findings on a common benchmark show that none of the five tested methods significantly outperformed the base model or fully reproduced its original results. This indicates that the rewards designed for these techniques optimize for format, length, or brevity rather than factuality and relevance.

Caveats

The study is a preprint that has not yet been peer-reviewed. The authors also note that evaluating factuality remains fragile across the field, as benchmarks lack ground-truth answers and rely heavily on retrieval and automated model judges.

From the paper

Limitation

Our common evaluation uses Llama-3.1-8B-Instruct with LoRA adapters, four English datasets, and smaller training budgets than the original papers.

Limitations

Quoted word for word from the full text on arXiv.

In this paper

reproducibility and fair comparison of factuality alignment methods

Methods
Flame (evaluated)FactAlign (evaluated)KnowRL (evaluated)FactGRPO (evaluated)Faith (evaluated)Llama-3.1-8B-Instructgemma-2bLlama-3-8B
Datasets
SimpleQA (evaluated)TruthfulQA (evaluated)FActScore (evaluated)Alpaca-Fact (evaluated)AlpacaEval (evaluated)
Links
Project pageWeights

Results

FactGRPO, relevance-filtered F1@K, biographies: .017 above the base model

Its relevance-filtered \text{F1}@K on biographies is.017 above the base model and is the only gain apart from the Faith artifact that is statistically significant, which is a small fraction of the published improvement.

5.2 Overall Results

Faith, TruthfulQA accuracy, on TruthfulQA: lose about 25; FactGRPO, TruthfulQA accuracy, on TruthfulQA: 18 points

From Table 2, we observe that Faith and FactGRPO lose about 25 and 18 points of TruthfulQA accuracy, while the other three methods stay within two points of the base model, which is not statistically significant.

5.2 Overall Results

What it shows

Under the common benchmark, no method significantly outperforms the untrained base model across the four evaluation datasets, except for two specific long-form cases.

We found that no method improves significantly over the untrained model on any of the four datasets, apart from two long-form cases discussed below.

5.2 Overall Results

On-policy reinforcement learning and offline preference optimization perform similarly on the common benchmark, with the alignment step barely changing the model from the supervised fine-tuning stage.

On-policy RL and offline preference optimization perform similarly on the common benchmark, and the alignment step barely changes the model from the supervised fine-tuning stage.

1 Introduction

Excluding zero-claim responses raises Faith's Alpaca-Fact precision from 0.101 to 0.781.

Zero-claim responses receive zero precision; excluding them raises Faith’s Alpaca-Fact precision from 0.101 to 0.781.

Extraction and scoring conventions.

Re-verifying biography claims against the subject's Wikipedia article lowers precision by about twenty points for methods producing long responses and changes their ranking.

Re-verifying biography claims against the subject’s Wikipedia article lowers precision by about twenty points for methods producing long responses and changes their ranking (Appendix C.4).

Evidence source.

Quoted word for word from the full text on arXiv.

The paper

Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison

Pratuat Amatya, Vinay Setty, V Venktesh, Avishek Anand

arXiv, Oct 13, 2026, Open access, CC BY, Preprint, not peer-reviewed

doi.org/10.48550/arxiv.2610.18415

arXiv:2610.18415v1 in cs.CL, also cs.AI

AbstractPDFHTML

Cite as BibTeX
@misc{amatya2026factuality,
  title = {Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison},
  author = {Amatya, Pratuat and Setty, Vinay and Venktesh, V and Anand, Avishek},
  year = {2026},
  eprint = {2610.18415},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  doi = {10.48550/arxiv.2610.18415},
  url = {https://arxiv.org/abs/2610.18415}
}

Related