Shailen Smith, Rasmus Torp, Adam Breuer
In this paper, we show that standard evaluations of high-resolution Model Inversion Attacks (MIAs) significantly underestimate training-data privacy leakage.In this paper, we show that standard evaluations of high-resolution Model Inversion Attacks (MIAs) significantly underestimate training-data privacy leakage.
Qizheng Zhang, Changxiu Ji, Isaac Sun et al.
Automated research systems increasingly run LLM agents over long horizons, but more inference does not by itself produce more progress: agents replay growing histories…Automated research systems increasingly run LLM agents over long horizons, but more inference does not by itself produce more progress: agents replay growing histories, duplicate one another's work, or stop experimenting while…
Yiran Zhang, Mo Zhou, Weihang Xu et al.
Learning Gaussian mixture models (GMMs) using the Expectation-Maximization (EM) algorithm and its gradient-based variants is a fundamental problem in machine learning.Learning Gaussian mixture models (GMMs) using the Expectation-Maximization (EM) algorithm and its gradient-based variants is a fundamental problem in machine learning.
Ziqun Bao, Xinyu Zhang, Yuchen Shao et al.
Safety auditing of post-trained large language models typically relies on model behavior, requiring model execution and depending on the coverage of available…Safety auditing of post-trained large language models typically relies on model behavior, requiring model execution and depending on the coverage of available evaluations.
Subham Rath, Raj Dandekar, Rajat Dandekar et al.
pass@k, the fraction of problems a model solves within k sampled attempts, is the field's default protocol for deciding whether reinforcement-learning (RL) post-training…pass@k, the fraction of problems a model solves within k sampled attempts, is the field's default protocol for deciding whether reinforcement-learning (RL) post-training on verifiable rewards improved a model.
Gowthamkumar Nandakishore
An open contrastive decision model is near chance as a judge on the hard public benchmarks: Contrastive-LM/CLM-v0.1-8B scores between 0.351 (best- of-four, chance 0.250)…An open contrastive decision model is near chance as a judge on the hard public benchmarks: Contrastive-LM/CLM-v0.1-8B scores between 0.351 (best- of-four, chance 0.250) and 0.593 (pairwise, chance 0.
Amartya Sanyal
We study mistake bounds for differentially private online learning and online prediction under oblivious realisable adversaries.We study mistake bounds for differentially private online learning and online prediction under oblivious realisable adversaries. Online learning requires the learner to release a hypothesis at each time step whereas in online…
Sophie L. Wang, Amil Dravid, Rulin Shao et al.
In this paper, we study how training data creates associations between the tokens at the start of a base model's response and the reasoning behavior that follows.In this paper, we study how training data creates associations between the tokens at the start of a base model's response and the reasoning behavior that follows.
Mohsen EsfandyariDoulabi, Lawrence Arkoh, Biruk Tadesse et al.
Predicting whether an LLM agent will fail has emerged as a promising direction for supporting intervention during execution.Predicting whether an LLM agent will fail has emerged as a promising direction for supporting intervention during execution. Recent approaches report strong predictive performance, often with AUROC values between 0.85 and 0.94.
Noor Munir, Francesco Quinzan, Stephen Roberts
Code large language model (Code LLM) assistants generate code from heterogeneous development contexts, including open files, imported modules, pasted snippets, and…Code large language model (Code LLM) assistants generate code from heterogeneous development contexts, including open files, imported modules, pasted snippets, and comments, much of which may originate from untrusted sources.
Ji Zhang, Zixin Liu, Yiran Ding et al.
An interaction has two parts: someone decides to act, and then chooses whom to act on.An interaction has two parts: someone decides to act, and then chooses whom to act on.
Beiming Liu, Minjie Chen
Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural networks.Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural networks.
Ferran Hernandez Caralt, Simon Heilig, Adrián Bazaga et al.
Empirical claims about the connection between over-squashing and long-range interactions in GNNs, can only be trusted if the benchmarks used to validate them genuinely…Empirical claims about the connection between over-squashing and long-range interactions in GNNs, can only be trusted if the benchmarks used to validate them genuinely require long-range interactions.
Steve Azzolin, Francesco Paolo Nerini, Stefano Teso et al.
Explainers for Graph Neural Networks (GNNs) are commonly evaluated by their plausibility, i.e., how well their explanations recover a predefined ground truth, such as a…Explainers for Graph Neural Networks (GNNs) are commonly evaluated by their plausibility, i.e., how well their explanations recover a predefined ground truth, such as a motif planted in the data.
Md Sazid Uddin, Md. Khairul Alam Mazumder, M. F. Mridha
Mechanistic edits (ablations, weight edits, activation steering) are the standard tools for unlearning a harmful capability from a neural network while preserving useful…Mechanistic edits (ablations, weight edits, activation steering) are the standard tools for unlearning a harmful capability from a neural network while preserving useful ones.
Hendrik Suhr, Sascha Xu, Jilles Vreeken
Scalable nonlinear causal discovery requires methods that combine flexible mechanism estimators with efficient search over large graph spaces.Scalable nonlinear causal discovery requires methods that combine flexible mechanism estimators with efficient search over large graph spaces.
Qiyuan Huang, Tianshi Xu, Meng Li
During post-training of large language models (LLMs) with Reinforcement Learning with Verifiable Rewards (RLVR), GRPO-style algorithms can exhibit severe late-stage…During post-training of large language models (LLMs) with Reinforcement Learning with Verifiable Rewards (RLVR), GRPO-style algorithms can exhibit severe late-stage collapse.
Pranay Kothari
Benchmarks for agent-generated data work grade a pipeline by running it once against a fixed snapshot.Benchmarks for agent-generated data work grade a pipeline by running it once against a fixed snapshot.
Jonathan Williams, Esin Tureci, Karthik R. Narasimhan
Majority voting over sampled completions is the workhorse of test-time scaling, and reinforcement learning with verifiable rewards (RLVR) is the workhorse for making each…Majority voting over sampled completions is the workhorse of test-time scaling, and reinforcement learning with verifiable rewards (RLVR) is the workhorse for making each completion better.
Xinyu Wang, Sicheng Lyu, Xiao-Wen Chang
Group-wise post-training quantizers for large language models round weights onto a grid that is not refit to the resulting integer codes.Group-wise post-training quantizers for large language models round weights onto a grid that is not refit to the resulting integer codes. We show that this leaves accuracy on the table: the best grid depends on the codes, input…
Langlin Huang, Hao Liu, Mononito Goswami et al.
On-policy distillation (OPD) has recently emerged as a promising post-training paradigm in which the student learns from trajectories generated by its own policy under…On-policy distillation (OPD) has recently emerged as a promising post-training paradigm in which the student learns from trajectories generated by its own policy under dense teacher supervision.
Yiming Huang, Yujie Zeng, Vijay Prakash Dwivedi et al.
Molecular learning models are strongly shaped by their underlying representations. Yet standard sequential and graph formalisms struggle to explicitly encode higher-order…Molecular learning models are strongly shaped by their underlying representations. Yet standard sequential and graph formalisms struggle to explicitly encode higher-order topology, such as ring systems and recurring motifs.
Michael Sullivan, Alexander Koller
Recent advances in LLM reasoning models—driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)—have enabled them to…Recent advances in LLM reasoning models—driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)—have enabled them to accomplish impressively complex tasks.
Haocun Ye, Xinlong Jiang, Qile Chen et al.
Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training.Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training.
Donghoon Lee, Shinjin Kang
A post-hoc out-of-distribution (OOD) detector reads the activations of a trained classifier and returns a score.A post-hoc out-of-distribution (OOD) detector reads the activations of a trained classifier and returns a score. It fits that score on in-distribution data, and the benchmarks that evaluate it supply a second piece of OOD data for…
Sushant Mehta, Logan Ritchie, Edwin Chen
Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior…Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an…
Haoyu Wang, Wei Zhao, Yedi Zhang et al.
Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or states in isolation.Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or states in isolation.
Juli Huang
Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are…Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior.
Shixuan Liu, Tongli Zhou, Junwei Deng et al.
Training data attribution (TDA) estimates the contribution of individual training examples to model outputs.Training data attribution (TDA) estimates the contribution of individual training examples to model outputs. Most scalable TDA methods rely on per-example gradients, whose computation and use at LLM scale pose challenges in…
Sohail, Sarkar, Shakuntala Baichoo
Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time.Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time. Its effect is reported as a scaling curve: accuracy against the number k of sampled answers.
Yuyang Wu, Yufeng Du, Hao Peng
Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distinguishing nearby…Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distinguishing nearby positions.
Jinmo Lee, Dooho Lee, Minho Jeong et al.
Can a pretrained graph model replace training and tuning a separate predictor for each dataset? Answering this requires evaluating prediction quality alongside…Can a pretrained graph model replace training and tuning a separate predictor for each dataset? Answering this requires evaluating prediction quality alongside computational cost.
Hung Phan, Thuy T. Nguyen, Minh Ngoc Dinh et al.
Time series foundation models (TSFMs) commonly adapt to new data by attaching a single trainable head to a frozen backbone, a one-size-fits-all setup that underfits…Time series foundation models (TSFMs) commonly adapt to new data by attaching a single trainable head to a frozen backbone, a one-size-fits-all setup that underfits heterogeneous regimes.
Juno Kim, Hengyu Fu, Peter Bartlett et al.
We compare the instance-wise, finite-sample risks of monotone spectral filters for linear regression, a broad class of estimators including principal component regression…We compare the instance-wise, finite-sample risks of monotone spectral filters for linear regression, a broad class of estimators including principal component regression (PCR), gradient descent (GD), and ridge regression.
Magali Parrino, Antoine Ajenjo, Emmanuel Remy et al.
Time series anomaly detection (TSAD) is increasingly deployed in streaming settings, where data arrive sequentially and may exhibit non-stationarity.Time series anomaly detection (TSAD) is increasingly deployed in streaming settings, where data arrive sequentially and may exhibit non-stationarity.
Xin Xu
Before a validator can be promoted to a hard gate on a deployment pipeline, it has to be shown that its firing separates outputs that reach users in working order from…Before a validator can be promoted to a hard gate on a deployment pipeline, it has to be shown that its firing separates outputs that reach users in working order from those that do not.
Haoran Tang, Rajiv Khanna
Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact…Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact its system prompt wants hidden, once the context rewards…
Yifei Wang, Xiaoyu Wu, Tsu-Jui Fu et al.
Diffusion distillation is widely adopted to accelerate sampling, and the resulting few-step models are broadly believed to match or even surpass their multi-step teachers…Diffusion distillation is widely adopted to accelerate sampling, and the resulting few-step models are broadly believed to match or even surpass their multi-step teachers in generation.
Fahao Chen, Linkang Du, Jinhao Zhou et al.
Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior introduces…Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior introduces previously unexplored privacy risks.
Ali Aouad, Aymane El Gadarri, Vivek F. Farias
Scaling laws for optimization against reward models in AI alignment have pinned down how performance depends on optimization effort—measured by a KL-divergence budget…Scaling laws for optimization against reward models in AI alignment have pinned down how performance depends on optimization effort—measured by a KL-divergence budget relative to a reference policy.