Search⌘ K
AI Features

Tool Abuse: Exfiltration, Agency, and Approval Bypass

Explore how to secure AI retrieval pipelines by implementing permission-aware retrieval boundaries, controlling access based on identity and entitlements, and preventing data leakage via caches or indexes. Understand the importance of provenance, auditability, and integrity checks to maintain safe AI system operations and prepare acceptance criteria for risk management and monitoring.

Prompt injection defenses assume untrusted content has already been fenced off from instructions. Retrieval is where that content actually enters the system, and if the boundary fails there, no downstream fence can fully compensate. Most retrieval-augmented generation incidents trace back to a failure at the retrieval boundary, not to a bad embedding. Permission-aware retrieval is the control that decides which evidence can enter the model context for a specific identity, tenant, and purpose. An index, a reranker, a cache, and a log all sit between identity and context packing. Together they form a security-critical control plane that can quietly bypass authorization or leak data through a side channel.

Retrieval is a trust boundary

A retrieval pipeline carries a clear security obligation. Prove every retrieved chunk was eligible for the requesting principal at the time of the request. Prove the resulting context got constructed without cross-tenant mixing. That obligation holds even when retrieval is fast, approximate, or layered. A vector index, a keyword index, a reranker, and hybrid search are implementation details; the boundary stays fixed. Evaluate identity and entitlements before any content is admitted into the candidate set that can reach the model.

A cache and a precomputed index create two common side channels. First, they can serve content computed under a broader entitlement set than the current request, turning staleness into leakage. Second, they can collide across tenants or roles when a cache key omits a relevant dimension: tenant_id, principal_id, role, policy_version, or purpose. Give the retrieval layer explicit cache constraints and invalidation semantics tied to an entitlement change event, not only to a time-to-live.

The following diagram shows the ...