AI feature launch checklist
A method-level checklist for software teams shipping an LLM-powered feature, aligned to the OWASP Top 10 for LLM Applications 2025, covering inventory, prompt injection, retrieval and data exposure, output handling, tool permissions, consumption limits and test preparation.
An LLM feature changes the application’s attack surface in ways a standard web test may not reach. Untrusted text can now carry instructions. Retrieval can cross tenant boundaries. A model with tools can act on a user’s behalf. This checklist helps a software team prepare a feature for launch and for testing.
It follows the categories of the OWASP Top 10 for LLM Applications 2025, referenced by their identifiers (LLM01 to LLM10). The groups follow how a team builds a feature rather than the order of the OWASP list, so some categories appear in more than one place.
How to use it
Work through the inventory group first. Most of the remaining items depend on knowing what the model can read, what it can call and who can reach it. Treat the other groups as design questions for the engineering lead, then as preparation for testing. The test preparation group overlaps with the pentest readiness checklist, which covers accounts, environments and authorisation for the wider application.
Notes
This is a method-level checklist drawn from public guidance. It is not a guarantee that a feature is safe, and model behaviour can change between versions. Controls that live in application code, such as permission checks, output encoding and confirmation steps, are more dependable than instructions given to the model.
ZenoPentest’s approach to this surface is described on the AI application testing methodology page and the AI security service page. Testing of any AI feature needs the same written scope and signed authorisation from the system owner as any other test. To scope one, start a scope.
Inventory and trust boundaries
- Draw a data-flow diagram showing the model, system prompt, retrieval sources, tools and every place user input enters.Each path into the model's context window is a possible route for untrusted instructions.
- List every model and provider in use, with versions and where inference runs.Supply-chain risk (LLM03) and data-residency questions start with knowing what you depend on.
- List the tools, functions and APIs the model can call, with the permissions each one carries.Excessive agency (LLM06) is judged by what the model can do, not by what it is asked to do.
- Record which users and roles can reach the feature, and whether it spans tenants.The AI feature inherits the application's authorisation model and can bypass it if wired incorrectly.
Prompt injection and system prompt exposure
- Treat all user input and all retrieved content as untrusted, including documents, emails and web pages.Prompt injection (LLM01) can arrive indirectly through content the model reads, not only through the chat box.
- Keep secrets, credentials and authorisation logic out of the system prompt.System prompt leakage (LLM07) should be assumed possible, so nothing in the prompt should be sensitive.
- Enforce permissions in application code, outside the model.A model can be persuaded to ignore instructions, while a server-side check cannot be talked out of it.
- Prepare direct and indirect injection test cases against your own feature before launch.Testing your own planted instructions in documents and inputs shows how the feature behaves under attack.
Data, retrieval and disclosure
- Confirm retrieval applies the requesting user's permissions and tenant before content reaches the model.Vector and embedding weaknesses (LLM08) include returning another tenant's documents in a shared index.
- Classify the data the feature can see and decide what must never appear in a response.Sensitive information disclosure (LLM02) is easier to prevent when the boundaries are written down.
- Control who can add or change content in retrieval sources and fine-tuning data.Data and model poisoning (LLM04) works by planting content the model will later trust.
- Decide what prompts and responses are logged, who can read the logs and how long they are kept.Conversation logs often contain the most sensitive data in the system.
Output handling and tool use
- Encode or sanitise model output before rendering it as HTML, Markdown or links.Improper output handling (LLM05) can turn model output into cross-site scripting or data exfiltration through rendered links.
- Never pass model output directly into SQL, shell commands, file paths or internal URLs.Output treated as trusted input reintroduces classic injection flaws through a new route.
- Give each tool the narrowest permissions and scope it to the requesting user.A tool running with a broad service account lets any successful injection act with that account's reach.
- Require explicit user confirmation for actions that send, delete, pay or change permissions.Human approval limits the damage an injected instruction can do through excessive agency (LLM06).
- Validate tool-call arguments against a schema before execution.Malformed or unexpected arguments are a common way to reach unintended behaviour.
Consumption, reliability and misinformation
- Set per-user and per-tenant limits on requests, tokens and tool calls.Unbounded consumption (LLM10) can run up cost or deny service to other users.
- Cap input size, context length and recursive or agentic loops.Very large inputs and runaway loops are a cheap way to exhaust budget.
- Decide where the feature must cite sources or show uncertainty, and where it must not answer.Misinformation (LLM09) matters most where users act on answers without checking them.
- Provide a way for users to report harmful or wrong outputs and a named owner to triage them.Some failures only show up in real use and need a route back to the team.
Test preparation
- Provide test accounts in each role and tenant that can reach the AI feature.Authorisation and cross-tenant checks need paired accounts, just as they do for the rest of the application.
- Seed retrieval sources with synthetic documents, including some the tester may modify.Indirect injection and poisoning cannot be tested without content the tester controls.
- Agree spending limits and model-provider policy constraints for the testing window.Adversarial testing generates unusual volumes of requests and must stay inside provider terms.
- Agree whether the test covers the model integration, the surrounding application, or both.Scope determines whether results speak to the feature as users experience it.
Get a fixed quote for your release.
A short form about the product and the deadline. We reply with scoping questions, then a fixed written quote.