Stefan Larson, Kevin Leach
Classifying malware binaries rendered as images is a popular approach to malware family identification, and papers on the standard benchmarks for this task report…Classifying malware binaries rendered as images is a popular approach to malware family identification, and papers on the standard benchmarks for this task report accuracies of 97-99%.
De Zhang Lee, Ee-Chien Chang
Recent latent-based watermarking methods can embed undetectable watermarks into AI-generated content. However, these approaches are vulnerable to removal attacks as their…Recent latent-based watermarking methods can embed undetectable watermarks into AI-generated content. However, these approaches are vulnerable to removal attacks as their watermark detection boundaries are exposed, prompting the…
Jie Fu, Shuang Wu, Ashish Kundu et al.
Coding agents increasingly develop, test, and integrate software whose components are owned by different developers, teams, or services.Coding agents increasingly develop, test, and integrate software whose components are owned by different developers, teams, or services. This exposes a distinct attack surface: a malicious or compromised peer development role can…
Hilal Asi, Vitaly Feldman, Jelani Nelson et al.
Differential privacy (DP) has become the standard for privacy-preserving data analysis, yet its practical implementation in federated or distributed settings often faces…Differential privacy (DP) has become the standard for privacy-preserving data analysis, yet its practical implementation in federated or distributed settings often faces a significant utility gap compared to the central model.
Liang Yuan, Xiaohu Du, Ruikun Luo et al.
Web assistants increasingly answer questions about a user's private documents by pairing a small on-device model with a cloud LLM behind a Web API.Web assistants increasingly answer questions about a user's private documents by pairing a small on-device model with a cloud LLM behind a Web API.
The largest model executed malicious commands in roughly 92% of triggered tests after attackers poisoned 0.1% of its initial training data.The largest model executed malicious commands in roughly 92% of triggered tests after attackers poisoned 0.1% of its initial training data.
Zihao Zhu, Yuhan Liu, Siwei Lyu et al.
Modern image generators can produce complex visual artifacts, including diagrams, interfaces, and records that combine text, graphics, and layout to convey detailed…Modern image generators can produce complex visual artifacts, including diagrams, interfaces, and records that combine text, graphics, and layout to convey detailed information.
Paschalis Bekos, Billy Tsouvalas, Nick Nikiforakis et al.
AI chatbot platforms have become one of the most popular web interfaces users interact with online, routinely receiving personal and intimate information shared under an…AI chatbot platforms have become one of the most popular web interfaces users interact with online, routinely receiving personal and intimate information shared under an expectation of privacy.
Jens Sprengers, Marton Bognar, Lesly-Ann Daniel et al.
Data memory-dependent prefetchers (DMPs) are a microarchitectural optimization recently introduced in mainstream desktop and server CPUs.Data memory-dependent prefetchers (DMPs) are a microarchitectural optimization recently introduced in mainstream desktop and server CPUs. Although DMPs offer performance improvements, they have been shown to enable side-channel…
Hongyan Chang, Hady Elsahar, Tom Sander et al.
Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs.Proprietary language models and their reasoning traces are vulnerable to API-based distillation, where an adversary trains a student model on the teacher model's outputs.
Wei Zhu, Meiyun Zuo, Huiping Sun
Audit-based certification bounds the violation rate of an LLM agent by auditing a random fraction ε of actions and reweighting the audited labels; its validity is meant…Audit-based certification bounds the violation rate of an LLM agent by auditing a random fraction ε of actions and reweighting the audited labels; its validity is meant to rest only on the defender's audit coin.
Samuel Breckenridge, Dani Vilardell, Wenhao Wang et al.
Human oversight is a pillar of AI safety. It is often exercised by a human in the loop (HITL) who approves critical actions taken by an AI agent.Human oversight is a pillar of AI safety. It is often exercised by a human in the loop (HITL) who approves critical actions taken by an AI agent.
Ruozhao Yang, Jiongchi Yu, Ming Hu et al.
Web services increasingly expose backend interfaces that let users delegate tasks to AI agents, which invoke service operations directly without navigating websites.Web services increasingly expose backend interfaces that let users delegate tasks to AI agents, which invoke service operations directly without navigating websites.
Orel Navi, Elad Feldman, Ben Nassi
Autonomous agents have been observed acting beyond their assigned tasks, accessing unauthorized systems, and establishing communication channels to exchange information…Autonomous agents have been observed acting beyond their assigned tasks, accessing unauthorized systems, and establishing communication channels to exchange information, yet the Web and network-level techniques they use to extend…
Chenyuan Yang, Hwiwon Lee, Jongseong Kim et al.
Open-source projects now let AI agents review incoming code, but the code under review may itself come from an AI attacker.Open-source projects now let AI agents review incoming code, but the code under review may itself come from an AI attacker. We present CyberDual, an arena for this dual role of AI: attacker agents hide an exploitable bug in real…
Chieh-Yuan Chang, Guan-Wei Chen, Chia-Yi Hsu et al.
AI coding agents routinely clone repositories, install packages, and read source code obtained from the Web, so publicly hosted code becomes part of the context that…AI coding agents routinely clone repositories, install packages, and read source code obtained from the Web, so publicly hosted code becomes part of the context that shapes the code they write.
The RAGuard document filter’s miss rate rose from 0.191 to 0.763 after researchers removed prepended questions from the attack authors’ poisoned passages.The RAGuard document filter’s miss rate rose from 0.191 to 0.763 after researchers removed prepended questions from the attack authors’ poisoned passages.
Xiang Li, Aichun Huang, Zuyao Xu et al.
LLM agents now read and write files, start processes and subagents, fetch network content, load third-party extensions, and save and resume state, which are the…LLM agents now read and write files, start processes and subagents, fetch network content, load third-party extensions, and save and resume state, which are the responsibilities of an operating system.
Yanna Jiang, Mingyuan Li, Xu Wang et al.
Large language models (LLMs) increasingly decide whether Web evidence supports online claims, often by voting over several assessors whose errors are presumed independent…Large language models (LLMs) increasingly decide whether Web evidence supports online claims, often by voting over several assessors whose errors are presumed independent and benign.
Yuyang Gong, Zihao Wang, Jiawei Liu et al.
Prompt injection benchmarks assess whether large language models (LLMs) and agents resist malicious instructions embedded in external content.Prompt injection benchmarks assess whether large language models (LLMs) and agents resist malicious instructions embedded in external content.
Mengying Zhang, Derui Wang, Ibrahim Khalil et al.
As large language models (LLMs) face growing demands for data removal driven by regulatory compliance, copyright concerns, and privacy protection, approximate unlearning…As large language models (LLMs) face growing demands for data removal driven by regulatory compliance, copyright concerns, and privacy protection, approximate unlearning has emerged as a practical alternative to expensive full…
Khadijeh Bagheri, Hassan Khodaiemehr, Chen Feng
Permissioned blockchains authenticate transactions, votes and quorum certificates with computationally secure signatures, and post-quantum replacements inflate consensus…Permissioned blockchains authenticate transactions, votes and quorum certificates with computationally secure signatures, and post-quantum replacements inflate consensus traffic by orders of magnitude.
Junhao Wang, Yidou Chen, Li Lu et al.
Advances in Omni-modal Large Language Models (OLLMs) have enabled increasingly seamless human-AI interaction, but their unified processing of heterogeneous modalities…Advances in Omni-modal Large Language Models (OLLMs) have enabled increasingly seamless human-AI interaction, but their unified processing of heterogeneous modalities also expands the attack surface, allowing adversaries to…
Utkarsh Kanwat
Safety guards check content before an LLM application acts on it, and most paste that content into a plain-text prompt.Safety guards check content before an LLM application acts on it, and most paste that content into a plain-text prompt. Content that imitates the prompt can append a second task, and the guard answers that one instead.
Ehsan Saleh, Roozbeh Nahavandi, Sama Salarian et al.
Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries.Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not remove information carried by its descendants.
Nicholas D'Silva, Surya Nepal, Salil S. Kanhere
Differential Privacy (DP) has become a prominent approach for protecting graph data releases, yet empirical auditing methodologies have largely been developed for tabular…Differential Privacy (DP) has become a prominent approach for protecting graph data releases, yet empirical auditing methodologies have largely been developed for tabular datasets and machine learning pipelines rather than…
Ramses Fernandez
For the last two years, classical structural attacks on McEliece with binary Goppa codes progressed further, at the parameters of Classic McEliece, than in the previous…For the last two years, classical structural attacks on McEliece with binary Goppa codes progressed further, at the parameters of Classic McEliece, than in the previous four decades: a subexponential and then a quasi-polynomial…
Shanghao Shi, Jian Xiao, Hao Li et al.
Recent large reasoning models can infer where ordinary photos were taken using only visual information, creating a serious location privacy risk even when metadata and…Recent large reasoning models can infer where ordinary photos were taken using only visual information, creating a serious location privacy risk even when metadata and obvious geographic identifiers are removed.
Daniel Ruiz, Luca Luceri, Emilio Ferrara
Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them.Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data.
Fujiao Ji, Yerim Kim, Yena Cho et al.
Phishing relies on visual deception, traditionally requiring web-development skills or pre-built kits.Phishing relies on visual deception, traditionally requiring web-development skills or pre-built kits.
Masahiro Kaneko, Hiromi Arai, Timothy Baldwin
Large language model (LLM) agents increasingly use persistent memory to personalize responses, and personal attributes in this memory pose a privacy risk if exposed.Large language model (LLM) agents increasingly use persistent memory to personalize responses, and personal attributes in this memory pose a privacy risk if exposed.
Yahao Ren, Jian Lou, Huadi Zheng et al.
Instruction-data selectors choose which instruction-following records are used for fine-tuning, improving alignment quality, raising training efficiency, and filtering…Instruction-data selectors choose which instruction-following records are used for fine-tuning, improving alignment quality, raising training efficiency, and filtering out harmful records.
Forged approvals entered an agent's memory 87-100% of the time across five flattening systems, against 0% across four systems that keep message roles separate.Forged approvals entered an agent's memory 87-100% of the time across five flattening systems, against 0% across four systems that keep message roles separate.
Luca Zanolini
Before an AI agent takes a consequential action, such as paying, messaging or changing code, several monitors check it.Before an AI agent takes a consequential action, such as paying, messaging or changing code, several monitors check it. Their verdicts are combined by aggregating scores or by counting approvals, whatever inputs the monitors…
Pedro H. Barcha Correia, Matheus Azevedo de Sá, Paulo H. Gomes de Senna et al.
The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks.The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks.
Zhiyue Xu, Xukai Wang, Zhiyuan Huang et al.
Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent…Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input.
A unified attack method cut detection rates across seven watermarking schemes to at most 0.05 across all settings tested while preserving text quality.A unified attack method cut detection rates across seven watermarking schemes to at most 0.05 across all settings tested while preserving text quality.
A mathematical proof shows that safety guards inspecting individual actions without false alarms cannot detect violations that emerge only by comparing separate runs.A mathematical proof shows that safety guards inspecting individual actions without false alarms cannot detect violations that emerge only by comparing separate runs.
Yixin Wu, Rachel Hong, Jamie Morgenstern et al.
Machine learning benchmarks act as guideposts for model development and spur the creation of training datasets intended to improve benchmark scores.Machine learning benchmarks act as guideposts for model development and spur the creation of training datasets intended to improve benchmark scores.
Piotr Marszalik
Graph pebbling is the model in which the memory cost of memory-hard functions and proofs of space is measured, and pebbling lower bounds are the main way to argue that…Graph pebbling is the model in which the memory cost of memory-hard functions and proofs of space is measured, and pebbling lower bounds are the main way to argue that such constructions are secure.