Jalil Alva, David Harper, Abigail E. Thornton et al.
Large language models are deployed globally, yet safety alignment and evaluation remain concentrated in English: harmful requests refused in English can elicit unsafe…Large language models are deployed globally, yet safety alignment and evaluation remain concentrated in English: harmful requests refused in English can elicit unsafe completions when translated into low-resource languages.
Xinying Cai, Bin Hu, Mingyu Jin et al.
Existing steering benchmarks are built around simplified, isolated generation tasks and evaluate a single behavioral attribute under controlled settings that do not…Existing steering benchmarks are built around simplified, isolated generation tasks and evaluate a single behavioral attribute under controlled settings that do not reflect the diversity and complexity of practical deployment.
Samee Arif, Furkan Danisman, Angana Borah et al.
Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level.Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level.
Albert Qin, Jonathan Cohen, Gautam Reddy
It has long been proposed that animals construct cognitive maps to support flexible behavior in novel environments.It has long been proposed that animals construct cognitive maps to support flexible behavior in novel environments. We tested whether large language models form similar representations using relational reasoning tasks and causal…
Antonio Lopardo, Luis Frentzen Salim, Shikhar Srivastava et al.
Tokenization is a critical component of the language modeling pipeline, but it is poorly understood. Tokenizers can impact downstream performance, but there is currently…Tokenization is a critical component of the language modeling pipeline, but it is poorly understood. Tokenizers can impact downstream performance, but there is currently mixed evidence about which tokenizer features predict a…
Ziqing Man, Di Wu, Jia-Chen Gu et al.
Context compression improves long-context efficiency but may introduce faithfulness issues through an information bottleneck.Context compression improves long-context efficiency but may introduce faithfulness issues through an information bottleneck. Existing evaluations rely on QA metrics, but a reader can still answer correctly from an unfaithful…
Shobhit Banga, Manas Dhir, Aaditya Pareek et al.
For most languages, automatic speech recognition (ASR) is limited less by model architecture than by a lack of accurately transcribed spontaneous speech.For most languages, automatic speech recognition (ASR) is limited less by model architecture than by a lack of accurately transcribed spontaneous speech. We present Monsoon, a framework for collecting such data at scale.
Abhinav Khattar, Aishwarya Chhabra, Ameya Velingker et al.
Building capable language models for India requires scalable multilingual data and robust evaluation. We introduce Nemotron-Pretraining-Indic-v1, a synthetic pretraining…Building capable language models for India requires scalable multilingual data and robust evaluation. We introduce Nemotron-Pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.83 trillion tokens covering all 22…
Howard Yen, Nitya Mani, Alex Renda et al.
Long-context language models (LCLMs) power agents that act over long trajectories, from improving a codebase to running research experiments to forecasters that condition…Long-context language models (LCLMs) power agents that act over long trajectories, from improving a codebase to running research experiments to forecasters that condition on lengthy histories.
Wei Gao, Cheng Tian, Duan Yang et al.
Deterministic hash-addressed conditional memory permits local post-hoc knowledge injection. Extending arXiv:2610.21553's bounded-feasibility audit rejecting…Deterministic hash-addressed conditional memory permits local post-hoc knowledge injection. Extending arXiv:2610.21553's bounded-feasibility audit rejecting under-specified generality claims with repaired probes and…
Bryan Li, Samar Haider, Grace Kim et al.
State media outlets often project their governments' narratives to target both domestic and foreign audiences.State media outlets often project their governments' narratives to target both domestic and foreign audiences. Because this content is plentiful and free to access, it appears widely in the web corpora that large language models…
Saeid Asgari, Rui Ying, Sara Malvar et al.
Checklist rubrics applied by large language model (LLM) judges have become a standard instrument for evaluating open-ended outputs and for rewarding them during training.Checklist rubrics applied by large language model (LLM) judges have become a standard instrument for evaluating open-ended outputs and for rewarding them during training.
Yue Huang, Zhangchen Xu, Xiaonan Luo et al.
We present 'Epi, an autonomous research agent that sustains investigations across model sessions and evolves its methods through accumulated experience.We present 'Epi, an autonomous research agent that sustains investigations across model sessions and evolves its methods through accumulated experience.
Anna Pramod
Small Language Models with quantisation and parameter-efficient adaptation are widely offered as the route to multilingual AI under compute constraints, and their…Small Language Models with quantisation and parameter-efficient adaptation are widely offered as the route to multilingual AI under compute constraints, and their efficiency is almost always reported as a single memory figure.
Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash et al.
We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception.We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception.
Peter Cho-Ho Lam, Ziyi Wang, Zirui Zhou
Prefix-tuning is a parameter-efficient fine-tuning method for large language models (LLMs) that steers model outputs by prepending task-specific prefix representations to…Prefix-tuning is a parameter-efficient fine-tuning method for large language models (LLMs) that steers model outputs by prepending task-specific prefix representations to the input sequence.
Haoxuan Jia, Yang Liu, Bin Chong et al.
LLM agents that read untrusted content can be hijacked by prompt injections. Runtime monitors that try to catch a hijack before the agent acts, and benchmarks that score…LLM agents that read untrusted content can be hijacked by prompt injections. Runtime monitors that try to catch a hijack before the agent acts, and benchmarks that score each task with a single run, share an assumption: whether an…
Husein Zolkepli
Forced aligners locate each word of a known transcript in the audio, and many speech pipelines depend on them.Forced aligners locate each word of a known transcript in the audio, and many speech pipelines depend on them. In our own data, a widely used CTC aligner inserts a pause at nearly every word boundary, and we found almost no…
Xin Qiu, Junlong Tong, Wei Zhang et al.
Large language models (LLMs) can now perceive, describe, and reason about images and audio, yet time series understanding remains underestimated: most work uses LLMs as…Large language models (LLMs) can now perceive, describe, and reason about images and audio, yet time series understanding remains underestimated: most work uses LLMs as forecasters and classifiers, reasoning studies are few and…
Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi et al.
Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls.Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls.
Aditya Nawal, Manit Baser, Mohan Gurusamy
LLM agents increasingly use retrieval tools across languages, yet their safety evaluation remains English-centric.LLM agents increasingly use retrieval tools across languages, yet their safety evaluation remains English-centric. It remains unclear whether language affects tool admission, post-retrieval generation, or both.
Prakhar Gupta, Terry Jingchen Zhang, Zhijing Jin
Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B." Sycophancy and related…Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B." Sycophancy and related biases show this failure.
Jianguo Huang, Zhiyi Lyu, Fuxiang Zhang et al.
Tree search enhances multi-step reasoning in large language models (LLMs), yet prevailing methods expand every step with a fixed budget, irrespective of the marginal…Tree search enhances multi-step reasoning in large language models (LLMs), yet prevailing methods expand every step with a fixed budget, irrespective of the marginal utility of expansion.
Anglin Cai, Zhiqi Wang, Xiaozhi Yang et al.
Multimodal large language models (MLLMs) increasingly serve as judges in training and evaluation pipelines, often assessing answers together with explanations.Multimodal large language models (MLLMs) increasingly serve as judges in training and evaluation pipelines, often assessing answers together with explanations.
Stephen Fernandes, Narayan Choudhary
Every multilingual speech pipeline begins with a language decision, an error there propagates silently through recognition, translation and routing.Every multilingual speech pipeline begins with a language decision, an error there propagates silently through recognition, translation and routing.
Yingfa Chen, Xingyu Shen, Chaojun Xiao et al.
Long contexts enlarge key–value (KV) caches, increasing memory use and attention costs during generation.Long contexts enlarge key–value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions.
Qiushi Sun, Jingyang Gong, Yang Liu et al.
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates.Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages…
Harsh Kumar
Bias audits of multilingual language models usually compare a model's average stereotype score in English with its average in another language and read the difference as…Bias audits of multilingual language models usually compare a model's average stereotype score in English with its average in another language and read the difference as a difference in fairness.
Jiahong Qian
Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures actually consist of…Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures actually consist of, and whether confidence signals can turn unreliable output…
Liang-Hsun Huang, Min-Yi Chen, Chao-Chun Chuang et al.
We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks.We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks.
Zhenyu Zhang, Zhichao Cao
An agent that drives an LLM sends the model the same conversation again and again, and each turn adds only a little text at the end.An agent that drives an LLM sends the model the same conversation again and again, and each turn adds only a little text at the end. Inference engines reuse the KV cache of the shared prefix, so only the new tokens pass through…
In tests across news and scientific abstracts, the same articles averaged 85% human or 15% human depending only on which prompt was on record.In tests across news and scientific abstracts, the same articles averaged 85% human or 15% human depending only on which prompt was on record. The researchers showed that these measures can be easily manipulated.
Xiaocong Yang, Yuren Hao, ChengXiang Zhai
We propose DAGFormer, a Transformer language model whose cross-layer connections are planned by a jointly trained topology predictor.We propose DAGFormer, a Transformer language model whose cross-layer connections are planned by a jointly trained topology predictor.
Adel Khorramrouz, Yi-Chia Wang, Sharon Levy
Social psychology has long described how people can be persuaded to interpret a harmful act as moral. Recent research suggests that Large Language Models (LLMs) echo…Social psychology has long described how people can be persuaded to interpret a harmful act as moral. Recent research suggests that Large Language Models (LLMs) echo aspects of human psychology, from cognitive biases to social…
Haoyuan Wu, Aoqi Wu, Jiani Hu et al.
Native multimodal pre-training trains a single model from scratch on text and multimodal data, enabling it to understand both language and visual inputs.Native multimodal pre-training trains a single model from scratch on text and multimodal data, enabling it to understand both language and visual inputs.
Danlin Li, Miao Li, Xu Han et al.
Layer-ablation studies often perturb early or late layers of a language model, compare the damage to prediction on two corpora, and read a larger early-layer effect on…Layer-ablation studies often perturb early or late layers of a language model, compare the damage to prediction on two corpora, and read a larger early-layer effect on one corpus as early-layer specialization for its task.
Jiwon Song, Yoongon Kim, Jae-Joon Kim
Sparse attention reduces the KV-cache traffic that dominates long-context LLM decoding, but practical methods largely operate at coarse granularity.Sparse attention reduces the KV-cache traffic that dominates long-context LLM decoding, but practical methods largely operate at coarse granularity.
Yirui Zhang, Yash Raj Shrestha
Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result.Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator.
Haolin Yang, Jipeng Zhang, Zhiqiang Qian et al.
Text-to-SQL models are commonly trained to map questions directly to static queries, whereas real-world database agents operate through stateful, multi-turn interaction…Text-to-SQL models are commonly trained to map questions directly to static queries, whereas real-world database agents operate through stateful, multi-turn interaction with live databases – inspecting schemas, executing probe…
Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh et al.
Long-horizon agents have drawn growing attention for their ability to work autonomously – from hours to days – on economically valuable tasks, a capability essential to…Long-horizon agents have drawn growing attention for their ability to work autonomously – from hours to days – on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI).