On Gemma 4 31B, one of 66 techniques beat budget-matched code selection on its primary target beyond the 95 percent uncertainty interval, consistent with chance.On Gemma 4 31B, one of 66 techniques beat budget-matched code selection on its primary target beyond the 95 percent uncertainty interval, consistent with chance.
Jize Guo, Xupeng Miao
The fastest machine-learning kernels are compositions of components, such as loaders, matrix-multiply mainloops, epilogues and communication stages, that run concurrently…The fastest machine-learning kernels are compositions of components, such as loaders, matrix-multiply mainloops, epilogues and communication stages, that run concurrently on one GPU streaming multiprocessor and share its on-chip…
Yaojian Chen, Dazhi Peng, Zhui Zhu et al.
A small number of GPU kernels often determines the efficiency of large language model training and inference.A small number of GPU kernels often determines the efficiency of large language model training and inference.
Marco Paina, Mark Niklas Müller, Niels Mündler-Sasahara et al.
Proactive bug discovery is an important software-engineering problem. Even though coding agents are already widely deployed in this domain, their capabilities remain…Proactive bug discovery is an important software-engineering problem. Even though coding agents are already widely deployed in this domain, their capabilities remain understudied.
Alexander Kovalenko, Artem Redchych
Small open language models can call tools, but they do not reason about what a call does: they delete without asking, repeat writes, re-read state they already hold, and…Small open language models can call tools, but they do not reason about what a call does: they delete without asking, repeat writes, re-read state they already hold, and issue one call per turn where a short program would do.
Srinath Srinivasan, Lunxiao Li, Tim Menzies et al.
Early stopping promises to reduce compute costs for autonomous coding agents by terminating unpromising runs early.Early stopping promises to reduce compute costs for autonomous coding agents by terminating unpromising runs early. Deciding when to abandon a run requires separating baseline issue difficulty from the trajectory of an individual…
Emmilly Immaculate Namuganga, Laurent Bindschaedler
Nine years after learned indexes were proposed, their advantage on read-mostly data is well established, and a standard benchmark rates their best concurrent version as…Nine years after learned indexes were proposed, their advantage on read-mostly data is well established, and a standard benchmark rates their best concurrent version as almost ready for use.
Weitao Feng, Hengheng Xiong, Zihao Zeng et al.
Large language models increasingly act through code, modifying repositories, executing programs, and interacting with services in agent form.Large language models increasingly act through code, modifying repositories, executing programs, and interacting with services in agent form. Their actions introduce risks that cannot be assessed from generated responses alone.
Saurav Bellam
Coding agents are given embedding search and code-graph tools on the premise that a repository's structure is something the model lacks and the tool supplies.Coding agents are given embedding search and code-graph tools on the premise that a repository's structure is something the model lacks and the tool supplies.
Yaorui Ding, Fengqing Jiang, Yichen Feng et al.
Training agents for front-end web development requires demonstrations of code editing, execution, browser inspection and revision, yet public trajectories and reusable…Training agents for front-end web development requires demonstrations of code editing, execution, browser inspection and revision, yet public trajectories and reusable task environments remain limited.
Sadra Emami, Benjamin Weigell, Mohammad Reza Mousavi et al.
Large Language Models (LLMs) are increasingly used to optimize code for performance. This practice often assumes that improvements in execution speed naturally translate…Large Language Models (LLMs) are increasingly used to optimize code for performance. This practice often assumes that improvements in execution speed naturally translate into overall lower energy consumption.
ZhiHong Liu, Shuyao Gao
Pipelines that combine large language models (LLMs) with static analysis let the model consume the analyser's output as evidence, but whether the model distinguishes true…Pipelines that combine large language models (LLMs) with static analysis let the model consume the analyser's output as evidence, but whether the model distinguishes true from false evidence, and whether its scrutiny depends on…
Antonio Ken Iannillo, Joshua Segun Owotogbe, Roberto Natella et al.
LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving…LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving agent resilience to realistic tool failures untested.
Lu Zhao, Christopher M. Poskitt, David Lo
Self-hosting coding agents is often necessary for software-engineering teams that must keep proprietary code and development traces within controlled infrastructure.Self-hosting coding agents is often necessary for software-engineering teams that must keep proprietary code and development traces within controlled infrastructure.
Avinash Maurya, Jaiaid Mobin, Jie Ye et al.
Long-context LLM serving for agentic, coding, and document workloads reuses prefixes whose key-value (KV) cache outgrows GPU and host memory, putting storage I/O on the…Long-context LLM serving for agentic, coding, and document workloads reuses prefixes whose key-value (KV) cache outgrows GPU and host memory, putting storage I/O on the critical path of the inference step for retrieving prefixes.
Julian Poeltl, Thomson Yen, Patrick Shea et al.
Spreadsheets are the lingua franca of modern finance, driving valuation, forecast, audit, and many other core workflows of the industry.Spreadsheets are the lingua franca of modern finance, driving valuation, forecast, audit, and many other core workflows of the industry. As LLM agents advance to automate knowledge work, spreadsheet construction is a natural and…
Jing Wei, Guanyi Chen, Qi Chen et al.
Data-intensive GPU applications increasingly rely on NVMe SSDs to extend GPU memory. NVIDIA GPU Direct Storage (GDS) enables direct SSD-GPU transfers via PCIe…Data-intensive GPU applications increasingly rely on NVMe SSDs to extend GPU memory. NVIDIA GPU Direct Storage (GDS) enables direct SSD-GPU transfers via PCIe peer-to-peer DMA, eliminating host staging.
Jianchang Su, Dawei Xiang, Yifan Zhang et al.
Depth pruning removes entire transformer layers to make large language models faster, and recent methods recover quality by replacing the removed layers with a…Depth pruning removes entire transformer layers to make large language models faster, and recent methods recover quality by replacing the removed layers with a training-free linear map.
Zhe Ye, Udaya Ghai, Zhizhen Qin et al.
Program optimization aims to improve performance while preserving behavior. Coding agents can already make programs faster by changing algorithms, data structures, and…Program optimization aims to improve performance while preserving behavior. Coding agents can already make programs faster by changing algorithms, data structures, and execution strategies.
Chenxi Huang, Haonan Wang, Jinghe Zhang et al.
Coding agents are increasingly capable of fixing real-world bugs, yet they still struggle with large systems codebases such as the Linux kernel.Coding agents are increasingly capable of fixing real-world bugs, yet they still struggle with large systems codebases such as the Linux kernel.
Ethan Seefried, Romit Ghosh, Sharmin Afrose et al.
Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure…Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling.
Jinze Li, Weijie Liu, Jinfeng Xu et al.
Software engineering (SWE) benchmarks are the primary way to measure the capabilities of coding agents: an agent is placed in a code repository or machine environment…Software engineering (SWE) benchmarks are the primary way to measure the capabilities of coding agents: an agent is placed in a code repository or machine environment, given a request, and judged by hidden tests.
Shihao Weng, Longtian Wang, Yang Feng et al.
Software quality has long rested on specifications that state what code must do, and on testing and verification that check that the code does it.Software quality has long rested on specifications that state what code must do, and on testing and verification that check that the code does it.
Soham Dan, Jeff Da, Sanyam Bhutani et al.
SWE-Bench Pro is a widely used benchmark of long-horizon software engineering tasks for coding agents. As agents improve, they reach task issues earlier agents never…SWE-Bench Pro is a widely used benchmark of long-horizon software engineering tasks for coding agents. As agents improve, they reach task issues earlier agents never touched and interact with the evaluation environment in ways the…
Quoc Co Tran, Hitesh Laxmichand Patel, Diwakar Mahajan et al.
AI agents are increasingly expected to modify and maintain machine-learning systems in production, yet existing benchmarks primarily evaluate either competition-style ML…AI agents are increasingly expected to modify and maintain machine-learning systems in production, yet existing benchmarks primarily evaluate either competition-style ML experimentation or generic repository issue resolution.
Jinbao Chen, Zixiang Zhou, Yu Zhang et al.
Static Application Security Testing (SAST) reports flag operations whose safety depends on project-specific context, and maintainers must decide which reports describe…Static Application Security Testing (SAST) reports flag operations whose safety depends on project-specific context, and maintainers must decide which reports describe real defects.
Mayank Arya, Agrim Bari, Anjaly Parayil et al.
Agentic large language model (LLM) applications turn one user request into a sequence of model and tool calls over a growing context.Agentic large language model (LLM) applications turn one user request into a sequence of model and tool calls over a growing context. Retaining this context's key-value (KV) state in GPU memory lets later calls reuse it instead of…
Yeounoh Chung, Jaehyun Ha, Helena Caminal et al.
Large-scale analytical query processing requires computing substrates capable of immense distributed scale-out.Large-scale analytical query processing requires computing substrates capable of immense distributed scale-out. Tensor Processing Units (TPUs) perfectly embody this extreme-scale architecture, offering massive computational…
Akhilesh Balasingam, Amadou Ngom, Geoffrey X. Yu et al.
Recent work has shown that LLMs can synthesize highly specialized database systems for fixed workloads, but these systems replace the DBMS's native storage and assume…Recent work has shown that LLMs can synthesize highly specialized database systems for fixed workloads, but these systems replace the DBMS's native storage and assume static data: users must migrate their data, often give up…
Seulkee Baek
Much of Ethereum's most consequential code is immutable bytecode: old output of unverified compilers, and protocol logic run as contract code.Much of Ethereum's most consequential code is immutable bytecode: old output of unverified compilers, and protocol logic run as contract code.
Di Wu, Xu He, Shu Wang et al.
While Large Language Models (LLMs) show promise for Automatic Program Repair (APR), existing benchmarks suffer from data contamination, in which test samples appear in…While Large Language Models (LLMs) show promise for Automatic Program Repair (APR), existing benchmarks suffer from data contamination, in which test samples appear in training sets, artificially inflating performance.
Anubhav Jangra, Smaranda Muresan
Code generation models are actively being used in professional and educational settings. However, the competence of large language models (LLMs) to generate…Code generation models are actively being used in professional and educational settings. However, the competence of large language models (LLMs) to generate algorithmically diverse code remains largely unknown.
Dipanwita Guhathakurta, Mert Toslali, Srinivasan Parthasarathy et al.
Deploying large language model (LLM) inference at scale requires jointly selecting hardware layout, parallelism strategy, scheduling policy, admission control, and…Deploying large language model (LLM) inference at scale requires jointly selecting hardware layout, parallelism strategy, scheduling policy, admission control, and request routing to meet latency and throughput targets.
At temperature 0, between 0.07% and 10.2% of rows changed their operator output over five executions, depending on the model.At temperature 0, between 0.07% and 10.2% of rows changed their operator output over five executions, depending on the model. A study of existing systems, reported in a preprint, measured how much semantic query engines vary…
Marcel Gregoriadis, Leonhard Balduf, Björn Scheuermann et al.
Data deduplication relies on content-defined chunking (CDC) algorithms to decide where data is split. We observe that most CDC algorithms are modular: a base chunking…Data deduplication relies on content-defined chunking (CDC) algorithms to decide where data is split. We observe that most CDC algorithms are modular: a base chunking algorithm fixes the byte-level cut-point condition, while…
Senad-Leandro Lemes Galera, Michail Georgoulakis Misegiannis, Ferdinand Gruber et al.
GPU working sets have outgrown GPU memory, so applications must oversubscribe it, keeping only hot data on the GPU.GPU working sets have outgrown GPU memory, so applications must oversubscribe it, keeping only hot data on the GPU. NVIDIA's Unified Virtual Memory (UVM) supports oversubscription, but its performance degrades once data exceeds…
Jiahao Wang, Hongjin Qian, Yuyang Hu et al.
As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows.As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows.
Mohammad Siavashi, Pascal Schwinte, Ali Banaei et al.
Agentic workloads resend long contexts that barely change between calls. Prefix caching now skips most of their prefill, yet every request must still be tokenized in full…Agentic workloads resend long contexts that barely change between calls. Prefix caching now skips most of their prefill, yet every request must still be tokenized in full before the cache can even be consulted.
Kazuki Egashira, Niels Mündler-Sasahara, Mark Niklas Müller et al.
Coding agents are increasingly used to create large software projects. While code-generation benchmarks have scaled accordingly to full applications, test-generation…Coding agents are increasingly used to create large software projects. While code-generation benchmarks have scaled accordingly to full applications, test-generation benchmarks remain limited to individual functions or bug fixes.
Mohammad Wathiq Soualhi, Basem Suleiman
Current software engineering agents are evaluated predominantly inside isolated sandboxes against synchronous local tests.Current software engineering agents are evaluated predominantly inside isolated sandboxes against synchronous local tests. However, deploying features in collaborative repositories requires modifying continuous integration (CI/CD)…