Search⌘ K
AI Features
Home
Courses
Machine Learning Security: Threats, Attacks, and Evaluation

The ML Security Threat Model Template

Explore how to create a structured threat model for machine learning security. Learn to define assets, attacker goals, access points, capabilities, constraints, assumptions, and trust boundaries. This lesson helps you make precise security claims and evaluate ML systems by filling in every detail in a threat model template.

We'll cover the following...

A threat model takes a single entry from an attack surface inventory and makes it specific enough to argue about: what exactly the attacker is after, what interfaces they can touch, and what constraints stop them. Let’s examine the difference that specificity makes when evaluating an ML system.

From inventory to threat model

Consider a vague threat model for our image-classification API: An attacker hacks the model and causes wrong predictions.

Now consider a structured threat model for that same system:

  • Asset: The deployed model artifact (model.pt) plus the POST /predict API outputs used by downstream automation.

  • Security objective: Prediction integrity for approved clients and service availability under expected traffic.

  • Attacker goal: Induce targeted misclassifications on a chosen class without triggering anomaly alerts.

  • Access point: Unauthenticated internet access to POST /predict (no access to the model registry or training bucket).

  • Capabilities and knowledge: Black-box query access; ability to send automated batches and observe confidence scores.

  • Constraints: Rate limits (100 req/min), request payload caps (5MB), and an operational query budget under $1,000.

  • Assumptions: Serving containers are patched, and registry access controls work as designed. ...