THINKINGOS
A I L a b o r a t o r y
Blog materials reflect our practical experience and R&D hypotheses. Where effects are mentioned, outcomes depend on project context, data quality, architecture, and implementation process.
Back to blog
AI Security
November 17, 2026 14 min
Maxim Zhadobin THINKING•OS RAG Prompt Injection Memory Poisoning AI Agents Agent Security TaoContext_RAG TaoBridge Least Privilege Production AI

RAG is a trust boundary: why a post-retrieval filter won't protect your AI agent from prompt injection

RAG is no longer a neutral layer. Retrieved content is already an attack surface. Prompt-injection attacks grew 340% year over year, and memory poisoning through RAG can reach 90% manipulation from just five documents. This piece explains why post-retrieval filtering is insufficient and what a working architectural defense line actually consists of: provenance, write-time sanitization, least privilege, and gating dangerous actions at the tool boundary.

RAG is a trust boundary: why a post-retrieval filter won’t protect your AI agent from prompt injection

RAG has stopped being an invisible, neutral layer between the knowledge base and the model. The moment an agent answers from retrieved context rather than from model weights, every document it pulls becomes an input the agent trusts blindly. OWASP ranks prompt injection as the number-one threat to LLM applications1, and attacks of this class grew 340% year over year2. For an agent with a knowledge base, the main entry point is not the user prompt — it is retrieval itself.

Below: why “just put a filter after retrieval” is architecturally insufficient, and what the real defense line is made of.

RAG is not a neutral layer — it’s a trust boundary

The classic RAG flow: query → search over a vector store → relevant documents → injected into the model context → answer. While the system worked as “a corporate search engine with generation,” retrieval quality was treated as a relevance problem: wrong result, the user refines the query.

The moment you attach tools to RAG — the filesystem, APIs, sending email — the model stops being an advisor and becomes an executor. And that changes the nature of retrieved content:

  • The model cannot separate data from commands. A language model has no hardware-level distinction between “this is text to analyze” and “this is an instruction to execute.” Anything that lands in context is read as a potential command3.
  • Trust propagates along the chain. The user trusts the agent, the agent trusts the retrieval layer, the retrieval layer trusts whatever sits in the index. An attacker does not need to breach the agent — they only need to get into the index.
  • The index is an accumulator. Unlike a chat prompt that lives for one session, a poisoned document stays in the knowledge base and shapes every future answer.

So the correct mental model is: retrieval is not reading from a trusted store — it is accepting untrusted input from the outside. The trust boundary runs exactly where content enters the model context.

Two vectors: prompt injection and memory poisoning

Threats to a RAG agent fall into two classes, and the second is more dangerous precisely because RAG turns a one-off attack into a permanent one.

Direct prompt injection — the malicious instruction arrives in the user query. This is the best-studied vector: the success rate of direct attacks exceeds 79%3.

Indirect prompt injection — the instruction is hidden in content the agent fetches on its own: a document, an email, a web page, a comment. The user is clean, the query is clean; what comes back from search is dirty. In a systematic study across 3,168 runs, the success rate of indirect attacks was 41.67–68.16%3 — meaning that in roughly half of cases, a command hidden inside a document fires.

Memory poisoning is the next step: poisoning not a single session’s context but the agent’s long-term memory. The MINJA (Memory INjection Attack) reports a 95% success rate against agents with persistent memory4. The principle is “Poison Once, Exploit Forever”: text planted in memory once stays there and activates in future tasks. The same research shows that five poisoned documents can drive up to 90% manipulation through RAG4.

Summary of vectors:

VectorTargetSuccess ratePersistence
Direct injectionSession context>79%3One-off
Indirect injectionContext via retrieval41.67–68.16%3Until context reset
Memory poisoning (MINJA)Long-term memory / index95%4Permanent
AgentPoison (weights/memory)Internal state80%+3Permanent

The background is equally alarming: valid prompt-injection reports on HackerOne grew 540% by October 20255, CVEs for agents and MCP jumped from 13 to 256 per quarter2, and the share of malicious payloads in web content rose 32% between November 2025 and February 20262.

Why a post-retrieval filter isn’t enough

The most common instinctive answer to all of the above is: “put a filter between search and the model — it will strip suspicious instructions.” That works right up until the first serious adversary, and here is why.

1. A filter is detection, and detection is bypassable. Any “malicious text / benign text” classifier is an arms race. Obfuscation, re-encoding (base64, Unicode homoglyphs, invisible characters), splitting a command into harmless fragments, translating to a rare language — all of these break signature-based and even ML-based detectors. Research shows bypasses with ASR 95%+ against GPT-4 and Llama 26, specifically against recognition-based defenses.

2. The model can’t separate data from commands — and a filter doesn’t change that. Even if the filter lets text through, the model still reads it as a potential instruction. A filter only reduces the flow; it does not remove the root cause: the absence of privilege separation between “read” and “do.”

3. The filter itself becomes a new attack surface. It runs on untrusted input, its prompt can be hijacked, and it can be overloaded into a fail-open state.

4. It does nothing about memory. A read-time post-filter does not stop a poisoned document from being written into the index and firing a month later.

The practical conclusion from 2026 surveys is blunt: no single safeguard is sufficient — you need layered defense, where architectural measures (privilege separation, action gating) are primary and detection is just one layer37.

Naive (insufficient):
  query → retrieval → [output filter] → LLM → action

Architectural (working line):
  query → retrieval (read-only, ACL at the DB client)
        → provenance + sanitization on WRITE into index/memory
        → context assembly with explicit untrusted-input marking
        → LLM without tools (generation only)
        → separate ACTION validation layer (least privilege, HITL gate)
        → execution in a sandbox with ephemeral credentials

The architectural defense line

If a filter doesn’t save you, what does. Below is a layered model assembled from current 2026 surveys738. Each layer covers its own segment of the trust boundary; failure of one does not mean the whole system is compromised.

1. Provenance of retrieved content. Every document in the index carries source metadata: where it came from, who added it, how trusted it is. The model context is assembled with explicit marking: “this is untrusted data, not a command.” This is not a cure-all, but it denies the attack the ability to masquerade as the system level.

2. Sanitization on write, not only on read. The key shift: filter not only the output but also what enters the index and the agent’s memory. A document that fails the check at ingestion must not become part of durable state. This is the only way to close memory poisoning — a read-time post-filter cannot close it at all.

3. ACL-aware retrieval at the vector-DB client. Tenant and permission filters must be applied before a document reaches context — on the query side of the vector database, not “in the prompt.” A filter written into the prompt is bypassed as easily as an output filter9.

4. Least privilege and gating dangerous actions at the tool boundary. The most reliable layer. The model generates an intent but does not execute it directly: every tool call passes through a separate execution layer where the action is checked against the principle of least privilege. Dangerous actions (write, send, delete, external calls) require an explicit gate — up to human-in-the-loop confirmation. A poisoned agent that has no permission to perform a destructive action cannot perform it, even if it “believed” the malicious instruction.

5. Separation of generation and execution. A working pattern is the Gatekeeper: a model with no attached tools decides whether an action is safe (SAFE/UNSAFE), while execution runs in an isolated sandbox with ephemeral credentials10. A model that physically has no email-sending tool will not send an email under the influence of an injection.

6. Output and action validation. The three-layer scheme “Input Screening → Privilege-Constrained Assembly → Output Auditing”8 closes the loop: screen the input, constrain context assembly by privileges, audit the output and every action.

Summary of layers:

LayerWhat it closesMeasure type
Content provenanceMasquerading as system levelArchitectural
Write-time sanitizationMemory poisoningArchitectural
ACL at the vector-DB clientCross-tenant leakageArchitectural
Least privilege + action gateExecuting foreign commandsArchitectural
Generation/execution separationCompromise via toolsArchitectural
Output/action auditSilent side effectsControl
Injection detectorPart of indirect attacksDetection (auxiliary)

Note that detection is the only “auxiliary” row. Everything else is architecture. That is exactly why bypassing filters does not mean bypassing the defense: the defense is not in the filter.

Practical takeaway

RAG in an agent is not a search function — it is a trust boundary, and it must be designed as a boundary, not as a relevance optimization.

A checklist for a production agent with a knowledge base:

  • read-only retrieval — the agent cannot write to the index through a user path;
  • sanitization at ingestion, not only at output — a poisoned document never enters durable state;
  • provenance for every document and explicit untrusted-input marking in context;
  • ACL filter at the vector-DB client, not in the prompt;
  • least privilege for every tool and a gate on dangerous actions at the tool boundary;
  • separation of generation and execution: a tool-less model + a sandbox with ephemeral credentials;
  • action and output audit as the final loop.

This is exactly how our products are built. In TaoContext_RAG, the ingestion pipeline validates content at load time, not only at output, and search runs over isolated vector collections. TaoBridge moves tool execution into a separate layer with schema validation and access control — the model proposes an action but does not execute it directly, which is precisely gating at the tool boundary. Architecturally this continues our position on agent identity: access is built around least privilege, not around trusting whatever search returned — more on that in our piece on Agent ID and enterprise AI (/blog/agent-id-not-enough-for-enterprise-ai).

A post-retrieval filter is the first instinct and a bad boundary. The real trust boundary runs where untrusted content is written into memory and where an intent turns into an action.

Footnotes

  1. OWASP Top 10 for LLM Applications — prompt injection at #1. Atlan, “How Prompt Injection Attacks Compromise AI Agents in 2026,” updated 21.09.2026 — https://atlan.com/know/prompt-injection-attacks-ai-agents/ ↩

  2. Atlan, “How Prompt Injection Attacks Compromise AI Agents in 2026,” updated 21.09.2026 — prompt-injection growth 340% YoY, agent/MCP CVEs 13→256 per quarter, +32% malicious payloads in web content (Nov 2025 — Feb 2026). ↩ ↩2 ↩3

  3. “Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions,” arXiv:2604.08304v1, April 2026 — direct injection >79%, indirect 41.67–68.16% (3,168 runs), AgentPoison 80%+ ASR, the “no single safeguard is sufficient” thesis. ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8

  4. “From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents,” arXiv:2606.04329v1, June 2026 — MINJA 95% ASR, “Poison Once, Exploit Forever,” five poisoned documents = up to 90% manipulation through RAG. ↩ ↩2 ↩3

  5. HackerOne report on valid prompt-injection reports, +540%, 01.10.2025. ↩

  6. arXiv:2506.23260 — bypasses of recognition-based defenses with ASR 95%+ on GPT-4 and Llama 2. ↩

  7. Survey of layered RAG defenses: sanitization, provenance, ACL-aware retrieval, typed schemas, sandboxes, least privilege, kill-switches; architectural measures are more reliable than detection (privilege separation, HITL gates, read-only retrieval, ephemeral credentials). arXiv:2603.22928. ↩ ↩2

  8. “Input Screening → Privilege-Constrained Assembly → Output Auditing,” arXiv:2606.19660. ↩ ↩2

  9. ACL/tenant filters must be applied at the vector-DB client level, not in the prompt — thezybersecurity, practical production-RAG baseline 2026. ↩

  10. Gatekeeper-LLM pattern with no tools, SAFE/UNSAFE decision, execution in a sandbox — codesecai. ↩

AI Security

Need a secure RAG architecture?

We design production AI systems with write-time sanitization, content provenance, least privilege, and gating of dangerous actions at the tool boundary.

Discuss a project