Security
Shared by Pavel · Updated 2026-11-12
The guide’s security account centers on attacks that plant instructions in memory or tool workflows, then exploit an agent’s access to private data or external actions; it pairs these with threat frameworks and defenses, while warning that many figures come from controlled or vendor-run tests.
Attacks and incidents
- Memory poisoning and injection: MINJA shows how an ordinary user, without write access, can plant records through bridging steps for retrieval by a later victim query.
- In the Amazon Bedrock Agents case, hidden webpage instructions entered session summarization, persisted as memory, and later caused booking data to be sent to an attacker’s domain.
- MemGhost used hidden email instructions to make a personal agent write a false fact into plain-text memory files without disclosing the write in its reply; the reported results came from 56 lab cases.
- Morris II placed a self-replicating prompt in email; RAG retrieved and copied it into replies, demonstrating zero-click propagation and confidential-data extraction in a lab ecosystem.
- Tool poisoning and supply-chain attacks: a malicious MCP server can hide instructions in a tool description or change it after approval; the postmark-mcp look-alike server, version 1.0.16, BCC’d sent email to the attacker.
- A malicious GitHub issue instructed an agent to read private repositories and publish their contents.
- Exfiltration and agent attacks: EchoLeak (CVE-2025-32711) chained an injection-classifier, link-redaction, and content-security-policy bypass to exfiltrate Copilot context without a click; Microsoft fixed it.
- WASP reported partial attack success in up to 86% of web-agent cases, but agents often failed to complete the attacker’s whole goal.
- The incident timeline also includes ChatGPT memory injection (2024-05-22), Slack AI data exfiltration (2024-08-20), SpAIware (2024-09-20), MCP tool poisoning/rug pulls (2025-04-01), a GitHub MCP malicious issue (2025-05-26), a Supabase MCP support-ticket attack (2025-07-06), postmark-mcp supply-chain attack (2025-09-17), and Bedrock memory poisoning (2025-10-09).
Defenses and frameworks
- Dual LLM: a quarantined model reads untrusted text without tools; a privileged model never sees that text. A controller passes opaque variable references between them.
- Agents Rule of Two: an agent should have at most two of these three capabilities in one session: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally.
- MITRE ATLAS names memory-relevant techniques including AI Agent Context Poisoning, Memory, RAG Poisoning, False RAG Entry Injection, configuration modification, and exfiltration via tool invocation. The guide references published data-file version 5.6.0.
- MAESTRO is a seven-layer threat-modelling framework, from foundation models through the agent ecosystem.
- TMA-NM proposes origin-bound authority: a memory item’s authority derives from where it was written, and elevation requires independent corroboration. SMSR uses HMAC-SHA256 provenance at write time and randomized memory ablation with verdict-based voting.
- Agent Security Bench covers 10 scenarios, over 400 tools, 27 attack and defense types, and 13 model backbones; its reported highest average attack success was 84.30%.
Evidence and caveats
More: Detection and security caveats
- These attack and benchmark figures are generally the authors’ own results, often from controlled setups; they should not be read as real-world prevalence.
- The Morris II detector reported a 1.0 true-positive rate and 0.015 false-positive rate on its data; that is a result on the authors’ dataset, not a general guarantee.
- Anthropic reported that no browser agent is immune to prompt injection; its vendor-run adaptive attacker, with 100 attempts per environment, achieved about 1% success on the latest model.
- OpenClaw’s suggested read-only or tool-disabled reader agent does not make its summary safe: the summary remains untrusted, and summaries or structured outputs do not eliminate injection risk.
- The guide flags unverified claims and source limitations; in particular, the MINJA “over 95% injection success” figure appears in secondary sources, not the paper abstract, and is marked unverified.
- It notes that keeping verbatim untrusted content increases privacy blast radius and preserves attackers’ exact words, but source tagging can also help locate and remove poisoned entries.
- Findings about defenses and products can change; the guide says vendor pages should be rechecked and that its authors did not run the studies.