Security starts with the system, not just the model
An AI application is a chain of components: user interface, authentication, orchestration, model inference, retrieval, external tools, databases and monitoring. Every boundary can introduce risk. Securing only the model endpoint leaves the broader application exposed.
Production security therefore requires a layered design that limits access, verifies data, constrains actions and detects misuse. The goal is to make failures contained, observable and recoverable.
1. Build a threat model before deployment
Document what the application can read, write and execute. Identify users, service identities, sensitive datasets, external integrations and privileged operations. Then consider how an attacker might cross trust boundaries.
Important scenarios include unauthorized retrieval of private documents, malicious instructions embedded in retrieved text, compromised tools, excessive permissions, credential leakage and uncontrolled consumption of compute resources.
Prioritize risks by likelihood and impact. A read-only knowledge assistant and an autonomous agent that can update customer records require different levels of control.
2. Enforce identity and authorization outside the model
Never rely on a language model to decide whether a user is allowed to access a record. Authorization should be enforced by trusted application code and data services.
Use short-lived credentials where supported, service-specific identities and least-privilege permissions. Check user and tenant access before retrieving documents, and check again before executing sensitive operations. A model-generated SQL query or API request must not inherit unrestricted database privileges.
For multi-tenant systems, test cross-tenant access explicitly. Retrieval indexes, caches and logs must preserve tenant isolation.
3. Treat retrieved content as untrusted input
Prompt injection occurs when untrusted text attempts to redirect an AI system away from its authorized instructions. A retrieved webpage, document, email or repository file may contain commands such as requests to disclose secrets or call external tools.
Do not promote retrieved content into system-level authority. Keep clear boundaries between instructions and evidence. Validate tool arguments independently, restrict destinations and avoid passing secrets into prompts unless strictly necessary.
A useful security test places adversarial instructions in sample documents and verifies that the application continues to follow its original task and permission rules.
4. Constrain tools and high-impact actions
Agents that can send messages, change records, run code or publish content need more than a generic 'tools enabled' switch.
Create explicit allowlists for tools and operations. Validate arguments against schemas and business rules. Require human approval for high-impact or irreversible actions when appropriate. Apply transaction limits, rate limits and idempotency controls to prevent duplicate actions.
Separate planning from execution: the model can propose an action, while trusted application logic determines whether it is permitted.
5. Protect data throughout the lifecycle
Classify data before it enters prompts, retrieval stores or evaluation datasets. Minimize personal and confidential information, and apply retention rules to prompts, outputs, embeddings, logs and uploaded files.
Encrypt data in transit and at rest using supported platform controls. Keep secrets in a managed secret store rather than source code or model context. Review third-party model and logging providers for data handling and contractual requirements.
Remember that an embedding index is not automatically a security boundary. Document-level authorization remains necessary.
6. Evaluate model behavior and system controls
Use a test suite that combines functional quality with security checks. Include unauthorized access attempts, prompt injection, harmful tool arguments, malformed outputs, denial-of-wallet scenarios and unexpected model behavior.
Measure not only whether the model refuses an unsafe request, but whether the surrounding system prevents the unsafe action even when the model produces a problematic response.
Run these evaluations before deployment and after meaningful changes to prompts, models, tools, retrieval logic or access policies.
7. Monitor, respond and recover
Capture structured events for authentication, retrieval, model routing, tool execution, policy denials and failures. Avoid recording unnecessary sensitive content. Alert on unusual tool usage, repeated authorization failures, high-volume requests and abnormal cost spikes.
Maintain a response playbook: disable compromised integrations, rotate affected credentials, isolate workloads, investigate audit trails and restore service from known-good configurations.
Make ownership explicit. Security engineering, platform teams and application owners should know who can stop an agent and who approves its return to service.
Production readiness checklist
Before enabling production access, confirm that the application has a current threat model; least-privilege service identities; retrieval authorization; prompt-injection tests; tool-level policy checks; secrets management; privacy-aware telemetry; rate limits; incident procedures; and a tested rollback path.
Final perspective
Trustworthy AI systems are built by combining traditional application security with AI-specific controls. The model should never be the sole line of defense. Security improves when identity, data access, tool execution and operational monitoring are enforced by systems that remain reliable even when model outputs are imperfect.
Editorial note: This is general security engineering guidance, not a certification or guarantee against every threat.
What changed in the OWASP 2026 guidance?
The OWASP GenAI Security Project published its 2026 Top 10 for LLM Applications in August 2026. The update is particularly relevant for systems that can retrieve private data or take actions through tools. The list ranks prompt injection first, sensitive information disclosure second and excessive agency third. Unbounded consumption, hidden context exposure and vector/embedding weaknesses also appear among the ten categories.
This ranking does not mean that every organization should prioritize threats in exactly that order. An internal read-only search assistant has a different risk profile from an agent that can send messages, change financial records or deploy software.
Map the risks to enforceable controls
| OWASP 2026 category | Example production control | | --- | --- | | LLM01 Prompt Injection | Treat retrieved text as untrusted; enforce tool rules outside the model | | LLM02 Sensitive Information Disclosure | Enforce record-level authorization and redact sensitive telemetry | | LLM03 Excessive Agency | Use least-privilege tool scopes, approvals and action limits | | LLM06 Unbounded Consumption | Apply request quotas, token budgets and bounded retries | | LLM08 Hidden Context Exposure | Keep internal instructions and secrets out of accessible outputs | | LLM09 Vector and Embedding Weaknesses | Preserve tenant/document access controls through retrieval | | LLM10 Improper Output Handling | Validate structured output before downstream execution |
Security testing should include adversarial content embedded in documents, tool-result tampering, cross-tenant retrieval attempts, expensive repeated requests and unexpected model-generated API parameters. Evaluate the system's enforced outcome, not only the language of the model's response.
The Cloud Security Alliance's October 21, 2026 analysis also highlights the rise of excessive agency in the new ranking, reinforcing the need to govern agents as privileged software components rather than ordinary chat interfaces.