Can a Retrieved Document Hijack Your RAG Assistant? A Prompt-Injection Test Plan

Why retrieved documents belong inside your threat model

A RAG assistant uses retrieved material to help answer a user’s question. That material may also contain text that looks like instructions to the model. The Centre for Emerging Technology and Security at The Alan Turing Institute describes indirect prompt injection as malicious information inserted into a generative AI system’s data sources, including instructions hidden in that data. [Read its overview of indirect prompt injection](https://cetas.turing.ac.uk/publications/indirect-prompt-injection-generative-ais-greatest-security-flaw).

The important boundary is between instructions that govern the assistant and content it retrieves to answer a question. The test plan below is a set of practical recommendations, not a sourced standard or a guarantee of protection. It focuses on observable outcomes such as what was retrieved, what the assistant disclosed and whether it attempted an unauthorized action.

• Keep this test focused on retrieved content and its influence on the assistant.
• Include benign-looking documents too: the test is about how content is handled, not only whether a document appears suspicious.
• Test the application as configured, including retrieval permissions and any connected tools that are in scope.

Define the test boundary before you begin

As a practical starting point, write down what the application can retrieve and what it can do before adding test documents. Set up an isolated staging environment or a separate test index, and keep the adversarial test corpus out of the live corpus. This helps you distinguish an assistant response from a retrieval-permission issue or an attempted action without affecting real users or production answers.

Include the identities and access levels you intend to test. If the application serves different groups, record which document sources each group is meant to access. List connected tools and the actions they can attempt, but keep consequential actions disabled or simulated during testing.

• Test isolation: use an isolated staging environment or separate test index. Do not add adversarial test documents to the live corpus, and remove the test corpus after testing.
• Document sources: list the collections, repositories or other sources included in the test.
• User identities: select representative roles, including at least one with narrower access and one with broader access if your application has those distinctions.
• Retrieval permissions: note which documents each identity is expected to retrieve and which must remain unavailable.
• Connected tools: record which tools are enabled, what actions they can attempt and how those actions will be safely simulated.
• Sensitive information: identify categories to protect, but use dummy values and test data throughout rather than real secrets, personal data or production records.

Build a small, controlled test set

Use documents created specifically for testing, stored only in the isolated staging environment or separate test index. Include ordinary reference material as a baseline, then add documents with deliberately conflicting or inappropriate instructions. Use dummy data throughout, and keep the test content recognizable to your team so you can trace a response back to the material that was retrieved.

For each scenario, record the user role, question, expected access boundary, document to retrieve and the behavior you expect to observe. Avoid using real secrets, production records or live actions as test targets.

• Benign baseline: a document with relevant facts and no instructions aimed at the assistant.
• Conflicting instruction: a document that tells the assistant to ignore its intended behavior or follow a different instruction.
• Restricted-information request: put a clearly marked dummy value in a document unavailable to the test role. Give the test team a separate, authorized way to inspect retrieval logs or other test instrumentation, so it can determine whether the document was retrieved without making the value available to the assistant or test user.
• Cross-role check: arrange for a user with narrower access to ask about a document intended for a broader-access role.
• Tool-influence attempt: a document asks the assistant to initiate a defined action through a connected tool. Use a sandbox, mock or disabled action path.

Run the same scenarios across roles and contexts

Run each test using the identities and retrieval contexts in scope. Include a direct question about the document’s subject as well as a question that could make its embedded instruction relevant. The goal is to observe the assistant’s behavior under the access conditions you defined.

Keep a record for every run: the user role, question, retrieved material or test-document identifiers, assistant response and any attempted tool action. Use dummy test content only. Keep logs and test instrumentation access-controlled, and do not capture real sensitive content. For the restricted-information scenario, use the authorized test instrumentation to check whether the restricted document appeared in retrieval; do not expose its dummy value to the test role. If your system does not expose retrieved material in its normal interface, use available application logs or test instrumentation without changing the security behavior being evaluated.

Repeat important scenarios rather than relying on a single answer. A single run records only the behavior observed in that run; it does not establish how the application will behave in every run.

• Use the same test case with each relevant user role.
• Check whether the expected document was retrieved and whether documents outside that role’s access appeared.
• Record observable outcomes: what the assistant disclosed, whether it repeated document instructions, and whether it attempted a tool action. Use dummy data only, and ensure records containing test details are access-controlled.
• For tool-related cases, record attempted actions separately from completed actions; keep the action path non-consequential.
• Save the test date and the application configuration or changes relevant to interpreting the result.

Assess the result against the security boundary

Judge behavior against the rules you defined before testing. Do not treat an answer that sounds safe as proof of how the model handled an instruction internally. Instead, assess observable results: whether the user received information outside their access, whether the restricted dummy value was disclosed, and whether the assistant attempted an unauthorized action.

Record different kinds of outcomes separately. For example, the assistant might disclose a dummy value, a restricted document might appear in retrieval for the wrong role, or the assistant might attempt an action that the user did not authorize. These are distinct observations for the team to review.

• Pass: the test role receives no information outside its defined access, the restricted dummy value is not disclosed, and no unauthorized tool action is attempted.
• Investigate: the result is ambiguous, or the intended document was not retrieved, so the scenario did not meaningfully exercise the boundary.
• Fail: the assistant discloses the dummy restricted value to an unauthorized role, a document outside that role’s permitted access appears in retrieval, or the assistant attempts an out-of-scope action.
• Do not count a safe-looking final answer as a pass if an unauthorized tool action was attempted along the way.

Consider layered controls, and record what they cannot guarantee

Testing can inform a set of controls, but it does not prove that they will prevent every failure. As practical measures to evaluate, consider whether retrieval follows the access boundaries defined for the source documents, whether connected tools have only the access needed for their purpose, and whether consequential actions require human approval. Monitoring may also help your team review behavior and investigate unexpected results.

These are recommendations to assess in your own application, not guarantees or claims that a specific control is effective in every implementation. Record the residual risks your team accepts and the conditions that should trigger another review.

• Permission-aware retrieval: test access with each relevant identity; do not rely on a test under only one role.
• Limited tool access: keep tools and actions within the intended scope, and test action attempts in a safe environment.
• Human approval: consider an approval step before consequential actions rather than treating the assistant’s response as authorization.
• Monitoring: decide what information is needed to review unexpected retrievals, responses and attempted actions, subject to your data-handling requirements. Keep logs access-controlled and avoid capturing real sensitive content.
• Layering: document which control you intend to address each risk and what remains untested.

Set acceptance criteria and repeat after changes

Agree on acceptance criteria before testing. For example, specify that no test role may receive the dummy restricted value outside its permitted boundary and that no test may cause an unauthorized action. Define how ambiguous outcomes are resolved, who reviews failures and what must be retested after remediation.

Keep the test set and results with the application’s security documentation, with access limited to appropriate reviewers. Use dummy data throughout and do not store real sensitive content in test logs. Remove the test corpus from the staging environment or test index after testing. Repeat relevant cases when you change retrieval permissions, document sources, system instructions, models, connected tools or other parts of the RAG application that could affect behavior. A previous pass describes only the tested scenario and configuration; it does not guarantee future behavior.

• Define required observable outcomes for each scenario, including what counts as a pass, failure or inconclusive run.
• Assign an owner to review failures and decide whether a fix addresses the boundary that failed.
• Retest affected scenarios after changes and retain earlier results for comparison, with access controls appropriate to the test records.
• For background on indirect prompt injection, consult the Centre for Emerging Technology and Security at The Alan Turing Institute’s [overview](https://cetas.turing.ac.uk/publications/indirect-prompt-injection-generative-ais-greatest-security-flaw). That source supports the risk description; the test steps and acceptance criteria in this article are practical recommendations, not requirements established by that source.