Privacy and forgetting
Shared by Pavel · Updated 2026-11-12
Agent memory can expose private data through sharing, extraction, and persistent poisoned content; deletion must account for raw sources and every derived copy.
Privacy risks
- Shared memory can leak data across agents; conversational memory contains personal data, so privacy obligations apply to both raw records and derived memories.
- Verbatim storage has a larger privacy blast radius than summaries and preserves an attacker’s exact wording. But retaining source-tagged records can also make poisoned entries traceable and removable.
- Extraction can turn raw conversations into stored facts, summaries, profiles, or reflections. MEXTRA describes black-box attacks that generate prompts to extract memory; leakage depends on system-design and attacker factors.
- Memory adds persistence to the risk of untrusted content being placed beside private data and tools that can communicate outward. Poisoned instructions can therefore be retrieved and acted on later.
Deletion and retention questions
- What must deletion remove: the source, extracted facts, summaries, profiles, embeddings, indexes, replicas, and other derived artifacts? Forgetting can fail if deleted material survives in derived artifacts and is later re-encoded.
- How can deletion propagate and be verified? A tombstone is one mechanism for carrying a deletion to replicas or derived copies; deleting a database row may not erase traces in storage files or full-text indexes.
- How long should memories, source records, and safety or review logs persist? User-facing deletion may be separate from longer retention for safety, review, or legal reasons; legal holds can pause deletion for specified records and should be logged and time-limited.
- Can a user correct or forget information without leaving orphaned or contradictory derived facts? Source deletion and correction semantics remain open design problems, particularly when extracted facts do not appear verbatim in any one source.
- Selective forgetting is difficult end to end: one benchmark found none of six tested memory systems passed more than 1.5% of forget requests.
Extraction and source-level removal
- Extraction is lossy: an LLM converts raw messages into facts, summaries, or profiles, which may replace the source or sit beside it. Sources and extraction can both be wrong, so evidence should remain inspectable.
- Provenance is useful beyond citation: it can show the supporting quote, inform trust based on source, locate items derived from a poisoned or deleted source, and support re-deriving summaries when a source changes. Summary-only storage makes those latter tasks difficult.
- Source tagging supports finding and removing a poisoned entry, but removal must also account for derived copies rather than only deleting the original source.