Security and Governance Closure: Controls and Evidence
Understand how to design and enforce security and governance controls with verifiable evidence in regulated generative AI systems. Explore threat modeling, trust boundaries, risk classification, control matrices, and release gates to ensure compliance and operational safety before deployment. Learn to manage residual risks and maintain oversight through ownership and review cadences.
The release pipeline from the last lesson gates a change on quality evidence. This lesson gates the same change on security and governance evidence. Reviewers approve regulated GenAI systems when high-risk controls produce verifiable evidence, not when artifacts list frameworks. The capstone risk and governance artifacts need an auditable chain for the highest risks: control placement, evidence signal, owner, and review cadence, plus how that chain is enforced at trust boundaries and checked at release gates.
Assets, actors, boundaries, and risk posture
Complete the threat model by identifying the assets that require protection and map them to the trust boundaries where security controls can be enforced. For a regulated GenAI system, key assets typically include user inputs, retrieved document chunks, tool outputs, system prompts and policy definitions, model and tool credentials, tokens, and secrets, audit logs, evaluation datasets, scores, and reports, and derived structured data persisted to downstream enterprise systems.
Actors and entry points matter because they determine where attacker-controlled data can cross into privileged execution. Common actors include authenticated end users, tenant administrators, internal operators, upstream systems that call the assistant API, downstream tools and services the assistant can call, and third-party dependencies in the model, retrieval, or tool ecosystem. Match trust boundaries to the prior boundary diagram, so controls land in enforceable places rather than in documents: the edge gateway that terminates user traffic, the retrieval boundary where queries reach the knowledge store, the containment envelope where tool calls are brokered, the identity boundary where principals and scopes are resolved, and the data egress boundary where responses and telemetry leave the environment.
Risk tolerance converts a long list of threats into two operational categories that drive design and operations. Must-reduce threats require preventive or strongly limiting controls ...