Apply least privilege
Give each agent a distinct identity and only the permissions needed for its assigned workflow. Avoid shared administrator credentials and broad access to entire databases. Separate read and write permissions where possible, and restrict access by resource, operation, and time.
Tools should expose narrow, well-defined actions. Validate inputs at the tool boundary and enforce authorization in the underlying service. Do not assume a model-generated request is safe simply because it came from an approved agent.
Use human approval for high-impact actions
Classify actions by impact and reversibility. Reading approved information may be low risk; sending external messages, changing customer records, approving payments, deleting data, or modifying production infrastructure may have significant consequences.
Require a person to review the proposed action and context before high-impact execution. Some actions may remain entirely outside the agent’s permitted scope.
Treat retrieved content as untrusted
Prompt injection can appear in user messages, documents, websites, or emails. Content retrieved by an agent should not override system policy or grant new authority. Separate trusted policy from external content, use allowlists where appropriate, and enforce authorization at the tool layer.
Test adversarial inputs that attempt to disclose data, change the task, or trigger unauthorized actions.
Protect data, observe behavior, and plan for failure
Keep API keys and secrets out of prompts and ordinary conversation context. Use secrets management, short-lived credentials where supported, and data minimization. Preserve user-level access rules when retrieving enterprise knowledge.
Log important requests, tool calls, authorization decisions, approvals, errors, and outcomes. Test incorrect tool selection, malformed arguments, unavailable APIs, stale data, repeated actions, unexpected costs, and attempts to exceed permissions. Define timeouts, safe retries, escalation, credential revocation, and rollback procedures.
OWASP’s 2025 LLM guidance highlights risks including excessive agency, prompt injection, and sensitive information disclosure. NIST’s AI risk guidance supports governance, evaluation, documentation, and oversight appropriate to the context. Security should be designed into the workflow before deployment.
Sources and further reading
OWASP Top 10 for LLM Applications (2025): https://genai.owasp.org/llm-top-10/
NIST AI 600-1, Generative AI Profile: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
NIST SP 800-207, Zero Trust Architecture: https://csrc.nist.gov/pubs/sp/800/207/final
