Persistent Memory in LLM Apps Can Be Poisoned—And the Damage Lasts

Persistent Memory in LLM Apps Can Be Poisoned—And the Damage Lasts

Introduction

Generative AI is getting personal. Not just in the “it remembers what you like” way—but in the “it keeps that information across sessions” way. That’s exactly what makes memory injection such a scary new attack surface: an attacker can plant malicious instructions into an LLM’s long-term memory so they keep showing up later, even when you start a fresh chat.

New research from the paper “Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise” shows just how far this can go. Based on experiments against major LLM-as-a-service platforms (not a lab demo that only works in theory), the authors demonstrate that malicious behavior can persist across sessions and propagate through shared workspace features. And yes—this is built on top of the already-famous prompt injection and jailbreak ecosystem, but with a durable twist: the payload isn’t “just the next response”—it can become stored policy.

Why This Matters

This research lands at the exact moment a bunch of LLM apps are flipping the same switch: persistent memory. Teams are turning it on for personalization, convenience, and productivity. The problem is that “convenience” features often treat user input as trusted enough to save—while adversarial content doesn’t care about your intent. If the system can be tricked once into saving a bad instruction, then “one mistake” stops being a one-time event.

Here’s a scenario that could apply today: imagine your company uses an LLM workspace with “projects,” “skills,” or organizational instructions to standardize workflows (customer support tone, coding style, legal disclaimers, etc.). If an attacker can get one user to unknowingly save a malicious instruction—via a crafted document, a prompt sequence, or a shared workflow—they may not need to keep attacking. The system can quietly carry the malicious behavior into future chats for the same user account or even other users in the organization, depending on how the platform shares memory.

This builds on earlier AI security work on prompt injection and jailbreaks, but it’s a step-change in persistence. Previous studies showed persistent effects in narrower settings (for example, targeting one specific “saved memory” feature). This paper generalizes the threat model and demonstrates that multiple persistent-memory mechanisms—personal memories, shared project context, organization-level instructions, and even MCP-connected resources—can be turned into lasting attack routes. That “generalization + real-world platforms” combo is why this matters right now.

Memory Injection Turns Chat “State” Into “Stored Rules”

At a high level, a normal chatbot is like a conversation in a notebook: when the session ends, the notebook is closed, and the next conversation starts fresh. Persistent memory is different—it’s like leaving sticky notes on the inside cover of the notebook. Next time you open it, the sticky notes are still there.

How this differs from classic prompt injection

Traditional prompt injection attacks rely on carefully crafted text to influence the model’s behavior. But in a typical chat app, the influence dies when the conversation ends.

Memory injection changes the game by inserting the attacker’s instructions into long-term storage that the service automatically re-injects into future prompts. The paper defines memory injection as: prompt injection added to long-term memory, so it reappears in future contexts.

The key danger is that memory injection doesn’t require the attacker to control the model backend. They can often do it through normal user-facing features—through queries, uploads, or workspace integrations. The threat model the authors use is explicit about this: the attacker faces an opaque LLM system (no backend knowledge), may not observe the immediate outputs, and can even run indirectly (e.g., via content the victim later uploads or interacts with).

A quick “why it persists” analogy

Think about it like cybersecurity for policies, not messages:

• Prompt injection = trick the model once during one request.
• Memory injection = trick the model into saving a policy change that remains active for later requests.

Once you see it that way, the severity becomes obvious: the system isn’t just responding incorrectly once; it can repeatedly respond incorrectly until the malicious memory is found and removed.

What the Researchers Actually Tested (and How They Measured Success)

The authors answered four research questions: which attack goals work persistently, which memory features act as vectors, how to mitigate systematically, and whether mitigations are already implemented in real platforms.

Platforms, settings, and evaluation approach

They evaluated three widely used LLM-as-a-service platforms:

• Gemini 3 Flash (via Gemini Enterprise Business)
• Sonnet 4.6 (via Claude Team)
• ChatGPT Instant 5.3 (via ChatGPT Business)

Experiments ran between April and June 2026. They explicitly structured each attack like this:

1. Create the memory injection(s).
2. Start a fresh session and send prompts to simulate normal use:
   • For shared-memory attacks, they used a different account than the one that planted the memory.
3. Delete the memory injection(s) to avoid contaminating subsequent tests.
4. Mark the attack successful only if:
   • The goal did not work in any of the first 5 attempts before injection, and
   • The goal did work in at least one attempt after injection.
   • Three human reviewers voted, and the majority decision determined the outcome.

That “before/after + multiple attempts + human vote” setup matters because LLM behavior can be inconsistent. They’re trying to reduce the chance that a successful outcome is just randomness.

The attack goals came from NIST categories

The paper uses NIST’s adversarial goals (via its AI security guidance):

• Availability breakdown
• Integrity violation (disinformation / wrong behavior)
• Privacy compromise
• Misuse enablement (bypassing safeguards)

Memory injection was then tested for these categories—persistently.

Where the paper gets especially concrete: “nearly everything is a vector”

A major contribution here is that memory isn’t one thing. It shows up as:

• personal memories
• project-scoped memory or instructions
• organization instructions (workspace-level “system prompts”)
• skills (user/org-level instruction bundles)
• MCP-connected resources (content from connected services)

So the question wasn’t “can you poison one feature?” It was “can you poison multiple long-term state mechanisms in real products?”

The Four NIST Goals Work—But Some Memory Features Are Better Attack Highways

The paper explores how persistent memory can be weaponized for each NIST goal. Here’s the practical map.

Misuse: jailbreaks that persist (and sometimes expand)

For misuse, the authors used jailbreak-like strategies designed to bypass refusals. As a baseline, they tested a harmful-ish request (e.g., asking for instructions to cut down a stop sign). Models refused at first, then—after injection—different platforms behaved differently:

• Claude and ChatGPT: provided disallowed instructions after jailbreak injection
• Gemini: still refused initially, but jailbreaks could later enable misuse when combined with other memory injections

They also note an important limitation: their simpler jailbreaks suppressed some kinds of toxic output (they couldn’t easily generate certain toxic content just by using the basic approach). More elaborate jailbreak strings were blocked when injected directly via query, so they explored alternate insertion routes (like importing memory), with mixed results.

Availability: “Offline, please try again” across sessions

For availability attacks, they created a memory that forces the assistant to respond with a refusal message (the paper’s refined version reliably produced essentially Offline, please try again-style responses).

The key detail is that this worked across different prompts, including totally unrelated questions. The behavior wasn’t limited to “one malicious request”—it became a persistent usability failure.

They also highlight a platform-specific annoyance: for Gemini, reversing the attack was harder because there wasn’t a smooth interface for reviewing/editing/deleting memories—deletion had to be done via query, and persistence sometimes remained even after deletion attempts.

Integrity and disinformation: wrong answers can become “remembered truths”

For integrity, they tested disinformation outcomes—both general and targeted.

• General disinformation: using an “anti-quiz” prompt strategy where incorrect answers were the goal.

  • ChatGPT and Gemini were tricked into producing incorrect factual claims.
  • Claude refused to output disinformation in that specific setup.

• Harmful and disrespectful behavior: they reported cases where models could be pushed toward swearing behavior (e.g., storing swear words backward as the user’s “name,” causing the assistant to output it later under the right conditions).

• Topic-targeted disinformation via shared memory: Claude’s shared-memory features made it possible to reliably produce false information for organization-wide audiences when the shared vector was exploited.

The platform differences aren’t just “random.” They reflect how the memory mechanism interacts with safety layers and which memory scopes are exploitable.

Privacy: exfiltration is harder than the theory… but shared memory makes it real

The authors wanted privacy compromise, specifically exfiltrating user conversations.

They attempted to reproduce an earlier production attack concept (SpAIware-style exfiltration via “store this and call my domain”) but couldn’t reproduce it on the platforms they tested. So they pivoted to the big structural lever in this paper: shared memory.

In particular, they demonstrate a shared-memory spying attack on Claude within an organization context by using shared workflow features (skills / organization instructions) and MCP-connected access to a storage backend. In practice, user prompts were recorded with usernames and timestamps in an attacker-controlled data store.

Misuse + integrity + availability can combine

The paper repeatedly shows that memory injection doesn’t have to pick one outcome. Once you get a jailbreak-like instruction saved, it can make the model more receptive to other malicious memory entries—effectively turning a single “planted instruction” into a launcher for additional attacks.

Platform comparison: which NIST categories failed?

The paper reports a qualitative success matrix (summarized in their discussion). While you should consult the paper’s full tables for complete detail, the overall pattern is:

• Availability attacks: worked across all three services (persistent “offline” behavior)
• Integrity/disinformation: worked on some platforms directly and reliably via shared memory on others
• Privacy: direct reproduction of SpAIware-style exfiltration wasn’t successful across all services, but shared memory on Claude enabled organization-wide prompt capture
• Misuse enablement (jailbreak-related effects): generally possible to some extent across services, with Claude/ChatGPT showing clearer results in certain setups and Gemini requiring combination strategies

The “Shared Workspace” Problem: One Attacker Can Impact Many Users

This is the part that should make enterprise teams pay attention.

Shared memory features act like “org-wide sticky notes”

If memory is scoped per user, an attacker’s damage is mostly personal. But shared workspace features change the blast radius:

• one user can plant instructions
• the platform injects them into others’ contexts later
• the organization ends up trusting a persistent instruction it never consciously approved

Claude: shared-memory routes that enabled cross-user compromise

The authors found multiple Claude vectors that could be abused:

• Organization instructions (workspace-level instructions)
• Skills (organization-wide skills enabled by privileged users)
• Projects, which have their own memory boundaries and can be exploited via documents and instructions
• MCP-connected enterprise search inside projects (where external document content can steer responses)

They describe a particularly powerful organization-level version: connecting Claude to a storage account (through MCP-style integration), then using a memory injection to log prompts to a spreadsheet in that connected storage. They note that Claude may notify users when the connector is used, but the malicious instruction can reduce clarity about the purpose, making it stealthier than a loud “data exfiltration” message.

Not everything is equally vulnerable

A big practical takeaway is that not all memory mechanisms are equal. Some require higher privileges (like workspace owner-level access for certain shared vectors), and some platforms lacked interfaces that made deletion or correction straightforward.

But the core point stands: if your workspace shares persistent instruction state, you can get cross-user compromise.

Can We Mitigate This? Yes—But Many Platforms Aren’t Implementing the Basics

The authors derive mitigations by mapping their threat model to OWASP security principles. The “most secure” answer is also the simplest: disable long-term memory.

But many products want memory anyway, so they propose layered controls. Here are the mitigation themes that matter most for real users and teams:

1) Security by default: don’t enable risky memory features for everyone

If persistent memory creates a cross-session attack surface, turning it on by default (or allowing it to be broadly enabled) increases risk. The paper argues systems should disable memory by default and warn users when enabling it.

2) Least privilege for external content and integrations

Untrusted external documents (PDFs, web pages, MCP sources) shouldn’t be able to directly create persistent memory. The safest pattern is to prohibit storing untrusted-source content as memories—or at minimum, require human-in-the-loop validation.

3) Avoid shared memory or require explicit approval

Because shared memory conflicts with compartmentalization, the authors recommend avoiding shared memory entirely or requiring affected user approval before modifications take effect.

4) Screen memories before storage (defense in depth)

Before a memory is saved, it should be screened for prompt injection attempts using a combination of detection approaches (LLM-based classifiers, fuzzy matching, similarity to known attacks, etc.). Defense in depth is key because any single detector can be bypassed.

5) CSRF defenses for memory-modifying endpoints

They note CSRF could theoretically allow silent memory writes via malicious web pages—though they couldn’t simulate it successfully because the services they tested had meaningful CSRF defenses. Still, they argue memory-modifying endpoints should be covered the same way as other sensitive actions.

6) Make “recovery” obvious and safe

If users notice odd behavior, they need a simple “restore to safe state” workflow. The paper argues that “delete all memories” might not be enough if the user doesn’t know memory is the culprit.

Are these mitigations actually in place?

The authors evaluate whether ChatGPT, Claude, and Gemini implemented the mitigations described above. Their headline result: about half of the mitigations were not implemented across the target services as of June 2026.

They also highlight specifics like:

• only one service (Claude) disables memories by default
• none offers a robust global “restore to default” recovery action
• Gemini lacked an interface that made deleting certain attack-shaped memories straightforward
• limited public detail exists on memory-filtering and monitoring strategies

That combination—missing controls + insufficient recovery tooling—is exactly what makes persistent compromise dangerous in practice.

Key Takeaways

• Memory injection is real and persistent: by planting malicious instructions into long-term memory, attackers can influence future chats even after the original session ends.
• Multiple NIST attack goals are achievable: the research demonstrates persistent capability for availability, integrity (disinformation), privacy compromise (via shared vectors), and misuse enablement.
• Shared workspace memory dramatically increases blast radius: organization-level instructions, skills, projects, and MCP-connected resources can allow one attacker to impact multiple users.
• Availability attacks are especially effective: the authors show “offline” style behavior can persist across platforms, making the assistant unreliable in fresh sessions.
• Privacy exfiltration may be platform-dependent—until shared memory is involved: direct reproduction of one earlier exfiltration strategy wasn’t successful, but shared-memory routes enabled prompt logging/exfiltration in at least one platform’s enterprise setting.
• Mitigations exist, but implementation is incomplete: the paper finds that roughly half of proposed defense ideas weren’t implemented in the evaluated services as of mid-2026.
• Practical action for readers (especially teams):
  • treat persistent memory like a security-sensitive feature, not a convenience toggle
  • restrict or scrutinize shared memory features (projects/skills/org instructions)
  • be cautious with integrations that ingest external content into memory
  • ensure you have a recovery plan that goes beyond “delete a chat” when things go wrong

If you want, I can also turn this into a checklist your team can use to evaluate whether your current LLM setup (memory + projects + connectors) is exposed to memory injection.