Changdae Oh, Qi Zeng, Qi Qi et al.
An emerging hypothesis about reinforcement learning (RL) post-training of large language models (LLMs) is that it merely sharpens existing behaviors of a base model…An emerging hypothesis about reinforcement learning (RL) post-training of large language models (LLMs) is that it merely sharpens existing behaviors of a base model, improving single-shot accuracy at the cost of solution coverage.
Stephanie Finley, Liudas Panavas, Thomas Mikkelson et al.
Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise…Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds.
Wensen Wu
ARC-AGI-3 evaluates agents in interactive environments whose rules and objectives must be inferred from observation.ARC-AGI-3 evaluates agents in interactive environments whose rules and objectives must be inferred from observation. We present Kepler, an open-source harness that represents hypotheses as executable world models and validates…
Michael Hardy, Ruhana Azam, Anka Reuel et al.
Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects of the evaluation…Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects of the evaluation conditions such as the scaffolds or tasks.
Katrina Drozdov, Oliver Chen, Langston Nashold et al.
Legal research is a core and time-consuming legal workflow. Lawyers must identify controlling authority, verify that it remains valid, reconcile statutes and cases, and…Legal research is a core and time-consuming legal workflow. Lawyers must identify controlling authority, verify that it remains valid, reconcile statutes and cases, and synthesize a grounded answer.
Sahan Paliskara, Nattaput Namchittai, Andrew Lampinen
People are increasingly delegating tasks to AI agents, and those agents are increasingly encountering other people's agents over shared resources such as a codebase, a…People are increasingly delegating tasks to AI agents, and those agents are increasingly encountering other people's agents over shared resources such as a codebase, a calendar, or a budget.
Daniel Dragonevskiy
Does a language model merely predict tokens, or does it understand what it says? We make this question measurable by defining "understanding" through the lens of…Does a language model merely predict tokens, or does it understand what it says? We make this question measurable by defining "understanding" through the lens of no-arbitrage.
Guanning Zeng, Jiani Wang, Wenjie Ma et al.
Learning to complete tasks in unfamiliar environments with unknown rules remains a key challenge for LLM agents.Learning to complete tasks in unfamiliar environments with unknown rules remains a key challenge for LLM agents. Current LLM agents often record their discoveries in prose, which may not provide a compact, explicit account of how…
Wenhao Liang, Lin Yue, Wei Emma Zhang et al.
Routing signals of modern vision transformers – expert gates, attention-residual weights and halting scores – often improve probes that predict whether the model is…Routing signals of modern vision transformers – expert gates, attention-residual weights and halting scores – often improve probes that predict whether the model is correct, and the improvement is commonly read as evidence that…
Joao Monteiro, Louis Béthune, Anastasiia Filippova et al.
As the context size of text processed with an LLM grows, the size of KV caches can outstrip the memory allocated for the original model weights.As the context size of text processed with an LLM grows, the size of KV caches can outstrip the memory allocated for the original model weights.
Yaxin Du, Xiyuan Yang, Zhifan Zhou et al.
Recursive self-improvement (RSI) seeks to enable AI systems to participate in improving their own capabilities.Recursive self-improvement (RSI) seeks to enable AI systems to participate in improving their own capabilities. A concrete pathway is autonomous model development, where agents iteratively explore post-training strategies to…