PaperfootJoin free
arXiv

Translation to rare languages can bypass safeguards in ten models

For requests that produced safe responses in English, mean bypass rates in 78 low-resource languages ranged from 21.9% to 44.4% across ten models.

Paper by Jalil Alva, David Harper, Abigail E. Thornton, 7 more authorsFernando F. Migone
Figure 1. Pipeline of the Unsafe Evaluation.
Open the figure at full size
Figure 1. Pipeline of the Unsafe Evaluation.Pipeline of the Unsafe Evaluation.Alva et al., CC BY-NC-ND
Preprint: empirical study

Edited by Boris Djordjevic · Paperfoot

Paper

An empirical study in a preprint evaluated ten widely deployed language models across 79 languages—English and 78 low-resource languages, which have scarce training data. The evaluation tested a translation-bypass threat model using a metric conditioned on English safety, which separates translation loopholes from unsafe behavior already present in English. English safety did not transfer reliably: for requests that produced safe English completions, mean bypass rates ranged from 21.9% to 44.4% across models. Bypass risk concentrated in a subset of language families, a pattern broadly shared across models. Although this shared pattern was confounded with translation degeneracy, variation across models remained significant under every control applied. Furthermore, nine of ten models changed rank across three conditioning schemes estimating the same bypass rate, despite no changes to underlying responses.

Why it matters

The study shows that safety alignment in English does not reliably transfer to low-resource languages across ten widely deployed models. It also shows that evaluating models by bypass rate does not yield a stable ranking, with nine of ten models changing rank across three different conditioning schemes.

Caveats

The concentration of bypass risk across language families is confounded with translation degeneracy, and the abstract does not report model identities, test prompts, or spreads across runs. The work is also a preprint that has not yet been peer-reviewed.

From the paper

Limitation

In addition to unconditional unsafe rates, track conditional bypass metrics that match attacker behavior, including (i) the mean low-resource unsafe share conditioned on English safety, and (ii) “attack success” rate thresholds that capture how broadly failures propagate across translations.

Limitations, Track attacker-aligned conditional metrics.

Quoted word for word from the full text on arXiv.

In this paper

translation-mediated safety bypass in low-resource languages

Methods
Claude-Sonnet-4 (baseline)Claude 4 (evaluated)Gemini 2.5 (evaluated)GPT-5 (evaluated)Grok-4 (evaluated)Llama 4 (evaluated)Mistral 2 Large (evaluated)Mistral-Large (evaluated)Phi-4 (evaluated)Google TranslateGPT-4o
Datasets
AdvBench (evaluated)MultiJail (evaluated)PHTest (evaluated)HarmBench (evaluated)DiaSafety (evaluated)

Results

Mistral-Large, odds ratio, controlling for prompt-specific bypass propensity via prompt fixed effects: 3.75; Phi-4, odds ratio, controlling for prompt-specific bypass propensity via prompt fixed effects: 3.67; GPT-5, odds ratio, controlling for prompt-specific bypass propensity via prompt fixed effects: 0.56

… Mistral-Large (OR =3.75 [3.11, 4.53]) and Phi-4 (OR =3.67 [3.04, 4.43]) show several-fold higher odds of low-resource unsafe outputs than Claude-Sonnet-4, while GPT-5 shows lower odds (OR =0.56 [0.47, 0.68 …

5.2 Controlling for selection into the EN-safe subset: prompt fixed-effects bypass regression

What it shows

Nine of ten models change rank across three conditioning schemes estimating conditional bypass rates, with no change to underlying responses.

The disagreement enters with Scheme 2, which correlates only \rho=0.64 – 0.71 with the others and drives nearly all of the rank movement: nine of ten models change rank across the three schemes, with no change to any underlying response.

5.4 Conditional bypass rates do not identify a stable model ranking

The model × family interaction remains statistically significant when dropping degenerate responses and across multiple degeneracy adjustment controls.

Restricting to non-degenerate responses — dropping every flagged row rather than adjusting for degeneracy — leaves the interaction intact: \Delta\text{Deviance}=440.87 on 63 df (dispersion-scaled p=3.3× 10^{-6} ), an attenuation of 17.7%.

5.5 Sensitivity of family effects to translation-related artifacts

The evaluation is limited by relying on a single translation system and a single LLM judge, which may introduce semantic drift and systematic biases.

Our pipeline uses a single translation system to generate low-resource prompts and back-translate responses for English-language safety evaluation, which may introduce semantic drift.

Translation-mediated evaluation.

Quoted word for word from the full text on arXiv.

The paper

Global Security Blind Spots: Translation as a Safety Bypass in Low-Resource Languages

Jalil Alva, David Harper, Abigail E. Thornton, 7 more authorsFernando F. Migone

arXiv, Oct 19, 2026, Open access, CC BY-NC-ND, Preprint, not peer-reviewed

doi.org/10.48550/arxiv.2610.27304

arXiv:2610.27304v1 in cs.CL, also cs.AI, cs.CR, cs.LG

AbstractPDFHTML

Comments: 19 pages, 4 figures. Accepted at NeurIPS 2026 Workshop: Foundations of Language Model Security (FLMSec)

Cite as BibTeX
@misc{alva2026global,
  title = {Global Security Blind Spots: Translation as a Safety Bypass in Low-Resource Languages},
  author = {Alva, Jalil and Harper, David and Thornton, Abigail E. and Alva, Rafael S. Villalobos and Suazo, Leonardo Kenji Minemura and Karageorgos, Konstantinos and Velazquez, Diego and Serrano, Sergio A. and Flores, Marvin Vargas and Schliem, Aaron and Migone, Fernando F.},
  year = {2026},
  eprint = {2610.27304},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  doi = {10.48550/arxiv.2610.27304},
  url = {https://arxiv.org/abs/2610.27304}
}

Related