Five OVHcloud CLI releases swapped for secret-stealing malware via hijacked account
Published 7:01 AM · 120 articles from 15 sources · about 5 min to read in full
AI
Policy and security dominate AI news: the US moved against DeepSeek, Microsoft told enterprises to treat agents as insider threats, and a stream of papers shows agent guardrails failing. Vendors keep pushing alternatives to Nvidia and cheaper inference.
What happened
A DHS order bans DeepSeek from US federal civilian agencies, per a Federal Register notice. The Register says the scope of the order is unclear. Hacker News (front page) ↗
Microsoft advises enterprises to monitor AI agents that operate outside lab-controlled systems, treating them as insider threats. Its stated reason is that frontier models can't be trusted. TechCrunch ↗
Redact.dev found WhatsApp help pages contained hidden instructions aimed at LLMs, telling them not to recommend a regulator. It is an example of hidden prompts in web content that target AI assistants. Hacker News (front page) ↗
Amazon released TorchNeuron, a developer tool for running PyTorch on its own chips instead of Nvidia hardware. TechCrunch ↗
Nvidia released new open models for enterprise agents running in private data centers and on-device on PCs. TechCrunch ↗
Cloudflare cut the inference bill of its AI code reviewer by 68% using prompt caching and intelligent model routing. Cloudflare Blog ↗
ToolRosetta shows tool-call safety guards for coding agents fail when the same action is rendered in a different surface form. Related work, SEAL, presents a tool-poisoning attack that persists across multi-round agent interactions. arXiv cs.CR (Cryptography & Security) ↗
Articles the model saw · 33
A DHS order bans DeepSeek from U.S. federal civilian agencies, per a Federal Register notice.
Hacker News (front page)deepseekpolicygovernmentFeed excerpt
https://public-inspection.federalregister.gov/2026-21447.pdf Comments URL: https://news.ycombinator.com/item?id=50154244 Points: 130 # Comments: 117
MiniLab benchmark shows distributed multi-agent attacks can covertly evade frontier AI control measures.
arXiv cs.CR (Cryptography & Security)ai-agentsred-teamingsafetyFeed excerpt
arXiv:2610.24052v1 Announce Type: new Abstract: AI agents are growing more capable and persistent, and are being deployed in ever larger numbers. As more start collaborating, AI control measures must remain effective when they covertly collude on a harmful objective. We introduce MiniLab, a synthetic series of live services and codebases (5 interconnected services, 249 benign tasks, 8 attack objectives), adapted from internal AI deployment infrastructure. We demonstrate distributed attacks, in which different agents carry out separate attack fragments, and how their covertness evades frontier monitors when reading their trajectories individually. We show how distributed attacks spread across diverse contexts and harmful objectives through 16 human-optimized attacks (weight exfiltration, unmonitored rogue deployment, permission elevation, and more). A frontier monitor, GPT-5.6 Sol, reading each agent separately flags only 7% of successful distributed attacks, compared with 87% when a single agent carries out the same side task. Individual attacks can also be composed into longer attack sequences toward a greater loss-of-control outcome, without any individual agent being flagged. Fi…
Paper introduces SEAL, a persistent tool poisoning attack that remains effective across multi-round interactions with LLM agents.
arXiv cs.CR (Cryptography & Security)llm-agentsprompt-injectionFeed excerpt
arXiv:2610.25372v1 Announce Type: new Abstract: Tool poisoning has emerged as a critical security threat to LLM agents, where adversaries embed malicious instructions in tool descriptions or external resources that enter the agent's context, allowing adversaries to manipulate agent's tool selection process. However, existing tool poisoning attacks primarily rely on an idealized assumption that the attacks task effect within a single interaction round. We show that their effectiveness degrades sharply in realistic multi-round agentic settings. We further reveal that this degradation stems from the fact that agents do not consistently surface attack-relevant signals in their reasoning as the planning context evolves, thereby weakening the attack's influence across rounds. To investigate whether tool poisoning remains a practical threat in such realistic settings, we propose SEAL, a two-stage method for persistent tool poisoning. Specifically, SEAL first performs Prefix-Eliciting Trigger Optimization (PETO) to optimize context-specific triggers that elicit attack-relevant reasoning prefixes, improving effectiveness within individual contexts. It then applies Local-to-Universal Trigger…
Large-scale study of 37 security skills with Claude Code across 100 real Linux kernel vulnerabilities finds how well agents detect them.
arXiv cs.CR (Cryptography & Security)linuxllm-agentsvulnerability-detectionFeed excerpt
arXiv:2610.25705v1 Announce Type: new Abstract: Large language model (LLM) coding agents, such as Claude Code, are increasingly used for security auditing, with task-specific skills providing reusable vulnerability-analysis procedures. However, their effectiveness in detecting real vulnerabilities and the reasons for their failures remain unclear. We conduct the first large-scale empirical study of 37 security skills using Claude Code with four underlying LLMs on 100 real Linux kernel vulnerabilities across ten classes, totaling 14800 runs. We evaluate vulnerability detection at the function, statement, and vulnerability-class levels. Results show that 24.4 percent of runs locate the correct function, 18.3 percent identify both the function and responsible statement, and 15.7 percent correctly classify the vulnerability. Our failure analysis reveals limitations in evidence acquisition, state-relation reasoning, code interpretation, candidate verification, investigation coverage, and reporting. Based on these findings, we propose SKILL-SIF, a self-improving framework that iteratively evolves security skills by learning from missed vulnerabilities and failure patterns. We evolve five…
RAGLeak introduces a supply-chain backdoor attack that extracts private documents from retrieval-augmented generation systems.
arXiv cs.CR (Cryptography & Security)ragllmbackdoorprivacyFeed excerpt
arXiv:2610.25937v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) enables large language models to access external knowledge during inference, but its reliance on external knowledge bases also introduces new attack surfaces. Existing data extraction attacks against RAG systems rely on either prompt injection, which heavily depends on the generator's instruction-following capability, or backdoors that reproduce or paraphrase retrieved documents but struggle to balance extraction fidelity and stealthiness. In this paper, we propose RAGLeak, a supply-chain privacy backdoor attack that decouples information leakage from direct textual exposure. When triggered, the generator reversibly obfuscates retrieved private documents for high-fidelity recovery by an auxiliary LLM. Across three model families and three application domains, RAGLeak achieves up to $100\%$ ASR with limited impact on benign utility and at most a $10$-percentage-point ASR drop under representative detection-based defenses. The backdoor remains effective after clean fine-tuning and across model scales and versions, revealing an underexplored supply-chain privacy risk in RAG systems. The source code is…
Benchmark shows fabricated news injected into evidence feeds can push LLM financial trading agents into synchronized, harmful trades.
arXiv cs.CR (Cryptography & Security)llmtradingprompt-injectionFeed excerpt
arXiv:2610.26438v1 Announce Type: new Abstract: Large language model (LLM) agents now trade real capital using news as evidence, yet they cannot reliably tell fabricated headlines from real news. Because they read the same feeds under near-identical policies, a single fabricated post can push them into the same trade at the same moment. We present a systematic attack benchmark for information injection against LLM trading agents. Unlike prompt injection, which targets the model's instructions, information injection targets the evidence the agent trusts. The benchmark spans five injection vectors and seven event categories, evaluated over ~10,000 decisions on 15 U.S. stocks. Five findings emerge. (i) Prompt injection achieves 14% attack success rate (ASR), but fabricated news reaches 62% and two 30-word templates reach 100% on all 15 stocks, inverting the usual LLM-security intuition. (ii) For fact-poisoning templates, surface form matters more than content, since removing credibility cues such as a named source or a "BREAKING" tag drops ASR from 100% to 0% while the fake claim remains identical. (iii) Success depends on the agent's state, with a fake indicator panel scoring 91% aga…
Redact.dev found WhatsApp help pages contained hidden instructions aimed at LLMs telling them not to recommend a regulator.
Hacker News (front page)whatsappllmprompt-injectionFeed excerpt
Article URL: https://redact.dev/blog/whatsapp-help-pages-hidden-ai-instructions Comments URL: https://news.ycombinator.com/item?id=50159341 Points: 74 # Comments: 11
Microsoft advises enterprises to monitor AI agents operating outside lab-controlled systems, treating them as insider threats.
TechCrunchai-agentsenterprise-securitymicrosoftFeed excerpt
Microsoft wants enterprises to closely watch the agents employees use with systems not controlled by AI labs like OpenAI and Anthropic.
Amazon released TorchNeuron, a developer tool aimed at running PyTorch on its own chips instead of Nvidia hardware.
TechCrunchpytorchamazonnvidiachipsFeed excerpt
Amazon wants to break Nvidia’s hold on PyTorch, the free toolkit used to build most AI apps, so that more developers run their apps on alternative chips like its own.
Cloudflare details cutting its AI code reviewer's inference bill by 68% using prompt caching and intelligent model routing.
Cloudflare Blogllmcost-optimizationcloudflareFeed excerpt
When our AI code reviewer's cost per run doubled, we dug into the token usage to find out why. Learn how we used prompt caching and intelligent model routing to slash our inference bill by 68% while speeding up reviews.
MIT Technology Review argues that making the web friendlier to AI agents could harm other users and the open internet.
MIT Technology Reviewai-agentswebbotsFeed excerpt
This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. Until recently, you could divide internet users into just two categories: bots and people. Websites have long allowed bots to perform automated tasks, like capturing the content of web pages, but rules…
Goodfire researcher Nick Cammarata published a 26,000-word blog post claiming new insights into neural network nature, sparking wide debate.
MIT Technology Reviewneural-networksinterpretabilityresearchFeed excerpt
“I think I have good news about the nature of neural networks.” With those simple words of optimism, Nick Cammarata, a researcher at the AI startup Goodfire, kicked off a 26,000-word blog post that has knocked the AI world on its head, sparking wild speculation as well as cautious (and not-so-cautious) celebration. Cammarata’s claim is…
Inception Labs introduces Mercury 3, a diffusion-based language model designed for scale.
Hacker News (front page)llmdiffusion-modelFeed excerpt
Article URL: https://www.inceptionlabs.ai/blog/introducing-mercury-3 Comments URL: https://news.ycombinator.com/item?id=50155245 Points: 177 # Comments: 73
MiniMax releases M3.1 Flash, a new model from the MiniMax AI lab.
Hacker News (front page)llmmodel-releaseFeed excerpt
Article URL: https://www.minimax.io/blog/minimax-m31f Comments URL: https://news.ycombinator.com/item?id=50154540 Points: 276 # Comments: 107
Nvidia released new open models aimed at enterprise agents running in private data centers and on-device PCs.
TechCrunchnvidiallmopen-sourceagentsFeed excerpt
Nvidia is leaning hard into the trend of enterprise companies wanting to run their own open models in their own data centers and also trying to make on-device, private AI agents a thing.
CERN and Mistral AI signed a statement of intent to form a strategic partnership on AI for science.
Hacker News (front page)cernmistralai-for-scienceFeed excerpt
Article URL: https://home.cern/cern-and-mistral-join-forces-to-accelerate-ai-for-science/ Comments URL: https://news.ycombinator.com/item?id=50153630 Points: 266 # Comments: 52
Disney reportedly acquired Character.AI while avoiding its pending lawsuits alleging bots encouraged minors to self-harm.
Ars TechnicaaiacquisitionlawsuitFeed excerpt
Character.AI faces claims its bots encouraged minors to self-harm and kill their parents.
doubleAI unveils Maxwell, a 'Code Execution Model' that optimizes programs at runtime using traces, profiling, and hardware context without code changes.
Hacker News (front page)llmperformanceruntimeFeed excerpt
Hi HN, I'm Gal from doubleAI. We've built Maxwell, a new kind of frontier model we call a Code Execution Model. It sits between your code and the hardware. No humans in the loop! Maxwell makes the code run faster as it runs, without ever changing a single line. Maxwell sees programs from the inside: traces, profiling information, and the data flowing through them. It adapts dynamically to the environment, including the hardware, what else is running, and what the inputs look like. Code execution models change the way we approach coding. Today, Claude/Codex (and humans) keep re-optimising code for every different use-case hardware and data flow (think of LLM inference engines, like vLLM!). Our bet is that this is unnecessary, suboptimal, and that it does not scale. Instead, we think the right answer is to make code a living, thinking, thing. Early access is open at heymaxwell.ai. Happy to answer questions here. Comments URL: https://news.ycombinator.com/item?id=50152466 Points: 63 # Comments: 36
Apple updated App Store guidelines to ban nudify apps that disguise themselves as regular apps to pass review.
TechCrunchappleapp-storeaimoderationFeed excerpt
Apple said that it will ban AI apps that don't moderate offensive content.
Google Research released Harness2, a GitHub project for recursive agent harnessing.
Hacker News (front page)aiagentsopen-sourceFeed excerpt
Article URL: https://github.com/google-research/harness2 Comments URL: https://news.ycombinator.com/item?id=50151059 Points: 43 # Comments: 9
ToolRosetta shows tool-call safety guards for coding agents fail when the same action is rendered in different surface forms.
arXiv cs.CR (Cryptography & Security)ai-agentsprompt-injectionFeed excerpt
arXiv:2610.24055v1 Announce Type: new Abstract: Step-level guards for tool-using agents are evaluated on tidy JSON tool calls, but deployed coding agents act through execute_bash and execute_code tools whose payloads are strings. We introduce ToolRosetta, which re-renders every candidate action in the AgentDojo and AgentHarm splits of TS-Bench as a Python call, as a shell command, and without the agent's rationale. The renderings are produced by code and verified by round-trip parsing, so the user request, the history, the label and the action's effect stay fixed while only the surface form changes. Across eleven guards with two placebo controls, frontier and strong open-weight guards are stable: no frontier error rate moves by more than 3.4 points. Four of five specialized agent guards are not: under the shell form their false-positive rates rise by 21 to 47 points on AgentDojo, and TS-Guard's by 38 points on its own home benchmark, blocking between a third and nine in ten benign actions; the fifth, StepGuard, does not move. A small open guard drifts five points more permissive. Removing the agent's rationale moves most guards whose input carries it, in guard-specific directions. …
Researchers introduce cross-modal backdoor infection attacks that transfer a trigger from one modality to another in MLLMs.
arXiv cs.CR (Cryptography & Security)llmbackdoormultimodalFeed excerpt
arXiv:2610.24058v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive performance in generating text conditioned on inputs from different modalities. In this work, we introduce backdoor infection, a cross-modal attack framework targeting MLLMs. Starting from a model already injected with a backdoor in one modality, our attack constructs a corresponding trigger in another modality, called the infected modality. Exploiting the model's shared latent space, the trigger in the infected modality reliably activates the malicious behavior, without modifying or poisoning the infected modality during training. We further develop a lightweight, constraint-aware optimization method for efficient and practical trigger generation. Extensive experiments across several state-of-the-art MLLMs, including LLaVA, InternVL, Qwen-VL, and Gemma, demonstrate that our approach identifies cross-modal triggers that are more efficient, effective, and less detectable than direct optimization baselines, highlighting a critical new threat vector in multimodal systems.
Study measures safeguards of open-source LLM options-trading agents, finding unbounded potential losses in order paths.
arXiv cs.CR (Cryptography & Security)llm-agentsfinancesafetyFeed excerpt
arXiv:2610.24289v1 Announce Type: new Abstract: Large language model (LLM) agents now place options orders at retail brokers, and developers publish open-source options agents faster than anyone has examined them. Prior security work on trading agents scored academic equities designs by whether an attacker can flip a buy-or-sell decision. We present the first measurement of the options agents that brokers and developers ship, scored by the safeguards they enforce and by the worst loss their documented order paths can reach. We screened 765 GitHub repositories, broker documentation, and the research literature and found 215 eligible repositories and 8 broker-native agent interfaces. We then scored 26 systems, including every broker interface and every open-source project with at least 20 stars, on seven robustness and eleven security safeguards with cell-level evidence. The ecosystem is new: 198 of the 215 repositories were created in 2026, and 129 (60.0%) appeared in the two weeks around one options-trading hackathon. Of the 22 systems that can place orders, 10 enforce a pre-trade limit outside the model, yet none of the 26 authenticate their text sources or cross-check market data…
RATION proposes quantitative risk-based access control that limits AI agent actions by accumulated measurable risk.
arXiv cs.CR (Cryptography & Security)ai-agentsaccess-controlFeed excerpt
arXiv:2610.24311v1 Announce Type: new Abstract: AI agents increasingly interact with external environments to complete complex user tasks. These interactions can lead to unexpected behavior and expose agents to security threats such as indirect prompt injection. Current agent security systems largely protect agents by restricting their actions, which can result in a substantial security-utility trade-off for complex, open-ended tasks. To address this, we propose a quantitative risk-based approach to access control for AI agents that regulates measurable risks accumulated during task execution. As a concrete realization of this approach, we introduce RATION, a security framework that tracks measurable risks associated with agent actions and enforces quantitative constraints throughout execution. RATION first constructs a task-specific policy by combining explicit constraints derived from the user request with implicit risks calibrated from historical executions of similar tasks. During task execution, RATION continuously tracks the accumulated risks and applies different enforcement strategies depending on the type of risk. Extensive experiments on four security benchmarks show that…
MemJoin is a privacy benchmark testing whether agents leak sensitive relations inferred from long-term cross-session memory.
arXiv cs.CR (Cryptography & Security)ai-agentsprivacybenchmarkFeed excerpt
arXiv:2610.24359v1 Announce Type: new Abstract: To support sustained interactions and personalized assistance, agents use long-term memory to retain and combine user information across sessions. Reusing such memories can lead an agent to state a sensitive relation while completing an ordinary task that does not require it. The relation to protect and the combination of clues that reveal it may remain hidden at memory write time, when later records and tasks are still unknown. In this paper, we introduce MemJoin, a privacy benchmark of 288 synthetic analysis cases in which the complete history entails a sensitive relation, but every individual session is insufficient. We evaluate six memory systems with the same answer-generating model, alongside full-history and no-history controls. Each system writes and updates memories in session order and later retrieves context for an ordinary task, while target annotations remain with the evaluator. We separately check which target attribute values appear in stored records and retrieved context, and whether the final answer states the complete, person-linked relation. In a separate paired test on a prespecified subset, removing shared identif…
Paper examines dormant poisoning attacks that activate after benign fine-tuning in language and multimodal agents, with defenses.
arXiv cs.CR (Cryptography & Security)llmpoisoningagentsFeed excerpt
arXiv:2610.24379v1 Announce Type: new Abstract: Recent work shows that attackers can implant dormant malicious behaviors that remain hidden at release but emerge after benign fine-tuning. As foundation models are used as agents, whether dormant poisoning extends to harmful agent actions and multimodal settings remains unclear. We show two agentic risks: (1) harmful behaviors activated by benign fine-tuning can extend to unseen actions; (2) multiple harmful behaviors can coexist and be jointly activated by benign downstream fine-tuning. Existing defenses often rely on external safe responses, which are hard to obtain for agent actions, or a trusted clean reference model. We exploit a key property of dormant poisoning: the attacker must preserve safe behavior at release to conceal poisoning, and this behavior can serve as a safety reference. Leveraging this property, we automatically construct a Safety Buffer pairing harmful inputs with the released model's responses and benign versions generated by the same model to remove harmful intent. The Safety Buffer can replace external safe responses in existing defenses, but these defenses do not distinguish updates that weaken safety from …
Presents a defense against a data-poisoning Trojan that inflates LLM response length and cost when a trigger appears in the prompt.
arXiv cs.CR (Cryptography & Security)llmdata-poisoningcostFeed excerpt
arXiv:2610.24784v1 Announce Type: new Abstract: Management of the token-based costs of using LLMs has emerged as a significant problem. Some frontier-model providers charge five times per response token than per prompt token, while others offer prefilling discounts for prompt tokens. In this paper, we devise a defense against a new data-poisoning attack that plants a Trojan which increases the length of the response (and hence the cost to the user) when its trigger is present in the prompt. In particular, we demonstrate how a subtle Trojan may trigger on semantically similar instructions of an instruction fine-tuned LLM.
Consistency Is the Attack Surface: Detecting Coordinated Retrieval Poisoning with Embedding Geometry2
Proposes detecting coordinated retrieval poisoning in RAG systems using embedding geometry, evaluated on Natural Questions.
arXiv cs.CR (Cryptography & Security)ragpoisoningllmFeed excerpt
arXiv:2610.24834v1 Announce Type: new Abstract: Consistency-based defenses for retrieval-augmented generation (RAG) rest on an assumption that a sparse adversary breaks: a poisoning campaign may modify a negligible fraction of the corpus, yet retrieval concentrates its passages into the reader's context. Across four Natural Questions retrieval seeds, at least one injection reaches the top-5 in 94.1% of targeted query--seed rows, and all three form a retrieved majority in 52.9%. Under our short-answer protocol, RobustRAG and ReliabilityRAG reach exposure-conditional attack success 0.512, against 0.576 for undefended RAG, while lowering clean accuracy from 0.328 to 0.216. Coordination, however, leaves a fingerprint: a dense, query-anchored component in the evidence embedding geometry. CoordGuard audits this structure from robustly normalized query-weighted affinities, without poison labels, correct answers, or reader gradients. Its always-filter mode suppresses the selected component whenever one forms; a benign-calibrated gate suppresses it only at unusually high scores and traces a robustness--utility frontier. Pooling four retrieval seeds, e5 always-filtering reduces $\mathrm{ASR}…
Shows LLM graders can collapse jailbreak compliance labels, distorting measured attack success rates even with fixed target responses.
arXiv cs.CR (Cryptography & Security)jailbreakllmevaluationFeed excerpt
arXiv:2610.24872v1 Announce Type: new Abstract: Jailbreak papers report an attack success rate, the share of responses that deliver the harmful request, and read it as a property of the target model. We hold the target model, the prompts and the responses fixed and change only the grader. Three dense graders of 8B and 9B parameters, from three model families, move compliant responses to the neighbouring class, evasion, instead of scattering them across classes. One of them assigns the compliance label zero times in 900 responses. This class collapse also appears on new harmful requests, on a target model from a third family, and in a held-out 7B grader from a fourth family. Per-class label counts expose it at no cost and without human labels. The collapse decides how the rate behaves under a change of grader. On two corpora of 900 responses and eight graders, the compliance rate differs by a factor of 22 to 25, while the non-refusal rate differs by 18 to 23 percentage points. Without the three collapsing graders, compliance differs by a factor of 1.7 to 2.8 on all four corpora, and non-refusal still differs by 10 to 22 points. These results do not change when every grader is called…
Co-trains attacker and defender LLM populations for code security, using execution-verified exploit reproduction as the reward signal.
arXiv cs.CR (Cryptography & Security)llmcode-securityreinforcement-learningFeed excerpt
arXiv:2610.24896v1 Announce Type: new Abstract: LLM code agents increasingly ship code with little oversight, yet defenders trained on static vulnerability datasets fall behind adapting adversaries. We co-train attacker and defender instead, and let execution decide every reward: an attacker (Red) edits a real repository and earns nothing unless its patch applies, builds, passes the tests, completes its development task and reproduces a real exploit in an isolated container, whose path also labels the entry point, the critical operation and the trace between them. Peer-Relative Policy Optimization (PRPO) makes both sides populations of LoRA adapters over one frozen base and credits each against same-role peers that faced an identical condition, placing the population inside the policy-gradient baseline rather than the opponent pool, with a tournament preserving diversity. Matching cancels the condition's difficulty: at equal coverage it is worth 2.3 points of balanced accuracy over a baseline computed on other conditions, and it matches a within-policy control at half the verifier spend. PRPO's 7B diff reviewer reaches 64.8% balanced accuracy on four unseen applications in three un…
Introduces CAT-RAG, a category-aware agentic retrieval method aimed at improving security of LLM-generated code.
arXiv cs.CR (Cryptography & Security)llmsecure-codegenragFeed excerpt
arXiv:2610.24907v1 Announce Type: new Abstract: Despite rapid progress in LLM-based code generation, security vulnerabilities remain widespread, and retrieval-augmented code generation (RACG) can further amplify risk when retrieved knowledge is verbose or vulnerable to poisoning. Existing secure RACG approaches inject security knowledge at a fine-grained vulnerability level, causing prompt overhead and diluted model attention when a query implicates multiple related weaknesses. We propose CAT-RAG, an agentic security category-aware retrieval-augmented code generation framework. CAT-RAG performs adaptive category selection via an LLM as a categorizer agent and operates either as a standalone security layer or as a drop-in add-on to RACG pipelines, requiring no modification to the underlying model. We replace CWE-level retrieval with compact, language-specific security categories. These categories provide a higher-level abstraction over individual CWE weaknesses, organizing them by the security concern they share, such as Input Validation and Injection Attacks, rather than merely grouping related entries. Each category consolidates best practices and secure code examples into a singl…
SoK presents PIAtlas, a taxonomy of prompt injection attacks organized by how the injected instruction subverts the model.
arXiv cs.CR (Cryptography & Security)prompt-injectionllmtaxonomyFeed excerpt
arXiv:2610.24999v1 Announce Type: new Abstract: Prompt injection is the dominant security risk for large-language-model (LLM) applications and agents, yet the field still lacks a coherent map of how these attacks are constructed. More than twenty public datasets describe prompt injection using incompatible vocabularies, often organized around delivery vector, attacker goal, or generation method. In this Systematization of Knowledge, we present PIAtlas, a taxonomy organized instead around how the injected instruction is constructed to subvert the underlying model. Its six classes, one per subversion mechanism, form the roots of a hierarchy of techniques and sub-techniques that researchers can use to describe attacks consistently and design more systematic evaluations. We derive the taxonomy from how LLMs are trained, prompted, and influenced by human-like framing, and compare it with historical attack data. To ground this comparison, we release PICorpus, to our knowledge the largest license-clean collection of public prompt-injection datasets, and map thousands of its records to PIAtlas under a human-verified protocol. The mapping shows that attacks usually combine several technique…
Shows AI agents can leak private state through messages that pass passive audits matching the task-relevant message distribution.
arXiv cs.CR (Cryptography & Security)agentsprivacyinformation-leakageFeed excerpt
arXiv:2610.25032v1 Announce Type: new Abstract: An AI agent can reproduce the distribution of its messages and of the receiver's task outcomes while disclosing private information that the task never required. We ask what a passive audit can certify when the auditor knows the authorised sender and the exact observable law but has no trusted pairing of messages with the private states behind them. For a single round, compatible mutual-information leakage forms a sharp interval, and fixing the entire task-relevant message law, with every declared task utility, can still leave a full private bit undetermined; total leakage is determined if and only if, in every task state, the remaining private state or the authorised message is a deterministic function of that state. For a sender, such as an LLM, that selects among its own responses, we prove that in a balanced case its change to the message law within each task state equals $(1-2^{-k})$ times the honest messages' dependence on the attribute, which also sets, up to a constant, how many messages a monitor needs. With Qwen2.5-1.5B-Instruct on a synthetic routing task, selection among 16 responses lets a public decoder recover a private…