Mohammadali Charoosaei, Cedric Richter, Mike Papadakis
Software is under constant evolution: developers continuously add features, fix bugs, and refactor code, and any of these changes may break existing functionality.Software is under constant evolution: developers continuously add features, fix bugs, and refactor code, and any of these changes may break existing functionality.
Shuangjie Yao, Hao Wang, Koushik Sen et al.
Large language model (LLM) agents are rapidly reshaping software engineering, accompanied by an explosion of new code benchmarks.Large language model (LLM) agents are rapidly reshaping software engineering, accompanied by an explosion of new code benchmarks. Yet nearly all existing benchmarks still rely on the same decades-old criterion: a solution is…
Haibo Jin, Peng Kuang, Xucheng Yu et al.
Large language models equipped with development environments have moved code generation toward repository-scale construction, yet building complete repositories remains…Large language models equipped with development environments have moved code generation toward repository-scale construction, yet building complete repositories remains difficult because interacting modules, interfaces…
Thiago Santos de Moura, Fynn Matuschek, Flavio Toffalini et al.
Large Language Models (LLMs) are widely used to generate code. Although their functional plausibility keeps improving, the generated code often contains security…Large Language Models (LLMs) are widely used to generate code. Although their functional plausibility keeps improving, the generated code often contains security vulnerabilities.
David Pissarra, Jinkun Lin, Haitian Jiang et al.
Optimized kernels such as FlashAttention and FlashDecoding are crucial for accelerating today's large models.Optimized kernels such as FlashAttention and FlashDecoding are crucial for accelerating today's large models. Most of them are handwritten by experts because existing ML compilers cannot match their efficiency.
Hyungyo Kim, Nicholas Satchanov, Hrishi Shah et al.
TRANSIT is a transparent scale-in framework to enable multi-node model training on fewer GPUs while maintaining training efficiency by transparently leveraging CPU DRAM…TRANSIT is a transparent scale-in framework to enable multi-node model training on fewer GPUs while maintaining training efficiency by transparently leveraging CPU DRAM as an extension of GPU memory during distributed training.
Arnab Kanti Tarafder, Jaume Guasch-Martí, Gokcen Kestor et al.
As Mixture-of-Experts (MoE) models scale toward hundreds of experts and higher top-k routing, memory efficiency in distributed training becomes a critical bottleneck.As Mixture-of-Experts (MoE) models scale toward hundreds of experts and higher top-k routing, memory efficiency in distributed training becomes a critical bottleneck.
Sarah McClure, Evyatar Cohen, Jakob Krebs et al.
The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing…The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing workloads that are sent at full line rates.
Édgar Chávez
In every fast nearest-neighbor index, recall is measured, never predicted: each operating point is tuned by serving it against ground truth.In every fast nearest-neighbor index, recall is measured, never predicted: each operating point is tuned by serving it against ground truth.
Andrew Geyko, Marius Mosbach, André Brinkmann
Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone.Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone.
Dolly Sah, Tanmay Sah, Harshul Jain et al.
Tool-using AI agents are increasingly deployed across enterprise software systems, yet widely used benchmarks primarily evaluate nominal task completion, conflating…Tool-using AI agents are increasingly deployed across enterprise software systems, yet widely used benchmarks primarily evaluate nominal task completion, conflating baseline planning competence with operational fault recovery.
Siyu Wang, Yifan Wang, Yuecheng He
Large language models are moving from producing trading signals to writing the code that executes them.Large language models are moving from producing trading signals to writing the code that executes them. The failure mode of the second role is silent: generated code runs, a backtest plots, yet the risk logic that the trader…
Yuexuan Xu, Jianyang Gao, Michael Norris et al.
SSD-resident approximate nearest-neighbor search is essential when vector collections exceed DRAM capacity.SSD-resident approximate nearest-neighbor search is essential when vector collections exceed DRAM capacity. The challenge is to reduce SSD reads and hide I/O latency through concurrent reads and overlap with computation.
Jianxing Qin, Alexander Du, Danfeng Zhang et al.
LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation.LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect system outputs.
Wei Song, Yuxin Cao, Xi Zheng et al.
Production LLM serving stacks combine an inference engine's local prefix cache with a shared KV-cache tier for fleet-wide reuse.Production LLM serving stacks combine an inference engine's local prefix cache with a shared KV-cache tier for fleet-wide reuse. The local cache distinguishes requests by adapter, weight configuration and sharing domain, but the…
Hantian Ding, Chloe Bi, Jiacheng Zhu et al.
Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories.Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories.