Check what your AI feature can read, reveal and do

The question we start with

Most AI security problems are not about clever prompts. They are about authority. An assistant that can read every document in a workspace will eventually show a user something they should not see. An agent that can send email with a service account will eventually send one nobody intended. So the first job is to write down what each AI feature can read, reveal and do, for each role and tenant, and compare that with what the product team believes.

That boundary map becomes the scope. It names the data sources, the retrieval layer, the tools, the model providers and the places where prompts are assembled. It also marks where each control is supposed to live. Controls that live only in the prompt are noted early, because they are the ones most likely to fail.

How we test

Testing follows the map. For each data source we check that retrieval is filtered by the application using the signed-in user’s rights, and that memory, conversation history and caches stay inside the right user and tenant. For each tool we check that it runs with the user’s permissions, that sensitive actions need confirmation, and that chains of actions are limited.

Prompt injection is tested as a question of consequence rather than a list of tricks. We introduce untrusted instructions through the channels your feature actually reads, such as uploaded files, retrieved pages or inbound messages, and observe whether the model can be led to cross a boundary: disclose another user’s data, call a tool it should not, or send output somewhere unsafe. Where the boundary holds, the report says so; that is evidence a reviewer will want.

The surrounding application gets the same attention as any web product: authentication on AI endpoints, authorization on admin settings, rate limits and cost controls, ingestion paths and logging. Public guidance on AI applications makes the same point: many serious issues sit in the ordinary application, not the model.

What good evidence looks like

Each finding shows the feature, the role and tenant used, the input channel, the observed behaviour and the boundary that was crossed. Evidence is recorded with synthetic data and documentation domains, and the report avoids reproducing anything that would be harmful outside your product. Fixes are written as design changes where possible: move a check from the prompt into the retrieval filter, add a confirmation step, scope a token.

What this assessment is not

It is not an evaluation of model accuracy, bias or general safety, and it does not test your model provider. It is not a certification against an AI regulation. It is a focused test of whether your product’s AI features respect the same data and permission boundaries as the rest of your application, with evidence your engineers and your customers’ reviewers can use.

Before you scope, the AI feature launch checklist covers the questions we will ask, and the AI application testing chapter explains the method in more detail.