Huang R, Xiong Y, Cheng YY et al.
Parameter-efficient fine-tuning (PEFT) methods adapt large models by introducing a small number of trainable parameters.Parameter-efficient fine-tuning (PEFT) methods adapt large models by introducing a small number of trainable parameters. However, we find that these parameters still contain redundancy and can be further compressed while…
Shilov I, Yang X, Panfilov A et al.
We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates.We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates.
Ou D, Lopez A, Missinne Q et al.
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through…Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments.
Reddy S, Abbas A, Suri A et al.
Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful.Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful. Because this safety-degrading data looks benign, standard content filters struggle to catch…
G\"urb\"uzbalaban M, Rafi MH, Aybat NS et al.
With exact, delay-free gradients, Nesterov's accelerated gradient (NAG) achieves optimal √κ dependence for L-smooth, μ-strongly convex objectives, where κ=L/μ; heavy-ball…With exact, delay-free gradients, Nesterov's accelerated gradient (NAG) achieves optimal √κ dependence for L-smooth, μ-strongly convex objectives, where κ=L/μ; heavy-ball (HB) achieves the same scaling on strongly convex…
Baniecki H, Biecek P, Pawelczyk M
Language model performance depends strongly on the mixture of domains in the pretraining data corpus. Popular data mixing methods select these domain weights by training…Language model performance depends strongly on the mixture of domains in the pretraining data corpus. Popular data mixing methods select these domain weights by training many small proxy models on random mixes and then fitting a…
Kim G, Hwang SJ
Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes.Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes.
Chang H, Elsahar H, Sander T et al.
Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs.Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs.
Cao S, Mao Z, Li D et al.
The rapid progress of software engineering agents drives interest in applying them to open-ended problems such as GPU kernel optimization.The rapid progress of software engineering agents drives interest in applying them to open-ended problems such as GPU kernel optimization. However, we find that existing coding agents converge to local optima on real-world…
Ziv L, Sharif M
While few-step distilled text-to-image models have gained traction due to their fast generation times, the speed-up implications on their unsafe generation rates remain…While few-step distilled text-to-image models have gained traction due to their fast generation times, the speed-up implications on their unsafe generation rates remain unknown.
Chen Y, Peng D, Zhu Z et al.
A small number of GPU kernels often determines the efficiency of large language model training and inference.A small number of GPU kernels often determines the efficiency of large language model training and inference.
Van MP, Tran C
A retrieval-augmented language model answers from two sources, the retrieved context and its own knowledge, and the two disagree whenever the context is incomplete or…A retrieval-augmented language model answers from two sources, the retrieved context and its own knowledge, and the two disagree whenever the context is incomplete or wrong.
Lehre PK
LLMs now prove hard theorems in Lean. Can they also design algorithms and prove their correctness and efficiency?LLMs now prove hard theorems in Lean. Can they also design algorithms and prove their correctness and efficiency? LLM-guided design of optimisation heuristics (such as AlphaEvolve) is purely empirical.
Hu B, Kociumaka T, Nogler J et al.
Tree edit distance (TED), the analog of string edit distance (ED) for hierarchical data, is the minimum number of node insertions, deletions, and substitutions…Tree edit distance (TED), the analog of string edit distance (ED) for hierarchical data, is the minimum number of node insertions, deletions, and substitutions transforming one labeled rooted ordered tree into another.
Cai W, Zhao J, Yuan Y et al.
Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly.Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly.
Xiaofeng Z, Lavoie S, Beltran-Velez N et al.
Discrete image generators learn the distribution of tokens produced by an image tokenizer. These token representations should be easy for generators to model, not merely…Discrete image generators learn the distribution of tokens produced by an image tokenizer. These token representations should be easy for generators to model, not merely enable faithful image reconstruction.
Knittel J, Helbling A, Pfister H
Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy.Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy.
Shi H, Wang J, Fu Y et al.
Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss…Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss small objects, text, and spatial relations or promote uncertain…
Jo E, Jang S, Mo S
Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved…Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved pages.
Zhang S
Dual-system vision-language-action (VLA) models pair a high-frequency fast policy with a slow, expensive module, and published trigger mechanisms for invoking it lack a…Dual-system vision-language-action (VLA) models pair a high-frequency fast policy with a slow, expensive module, and published trigger mechanisms for invoking it lack a shared objective, measured costs, and any standard separating…
Bian S, Choe H, Kim J et al.
We study a protocol-level reaction leakage in CKKS deployments where public coefficient-to-slot (C2S) evaluation keys are combined with a client's 1-bit validity…We study a protocol-level reaction leakage in CKKS deployments where public coefficient-to-slot (C2S) evaluation keys are combined with a client's 1-bit validity response.
Cicek U, Singh M, Shang Z et al.
While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate…While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate at the patch or region level, where annotation requirements…
Jafari A, Bhatia V, Qin M et al.
Speculative decoding pairs a cheap drafter with parallel verification against the target. At ultra-long context it stops paying, because a drafter that attends over the…Speculative decoding pairs a cheap drafter with parallel verification against the target. At ultra-long context it stops paying, because a drafter that attends over the full prefix inherits the target's context-dependent cost, so…
Chuang YS, Ge Y, Li Y et al.
A self-improving agent has to decide which new skills to keep, and in practice it keeps whatever earns a high reward, convinces a judge model, or compresses its library…A self-improving agent has to decide which new skills to keep, and in practice it keeps whatever earns a high reward, convinces a judge model, or compresses its library, even when the skill is wrong.
Yu Y, Jian Y, Li J et al.
Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising…Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps.
Feng Z, Hu L, Yuan H et al.
Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them.Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co-optimized with algorithm-system co-design, and present…
Zhao S, Zhou Z, Qi H et al.
Earth observation (EO) models are still built by hand. For each new task, experts design a training recipe and tune it against a benchmark.Earth observation (EO) models are still built by hand. For each new task, experts design a training recipe and tune it against a benchmark. We show that this loop can be handed to an agent.
Liu K, Li A, Li J et al.
We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding…We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage.