Query Access, Transferability, and Constrained Evasion Objectives
Understand how attackers leverage query access at prediction APIs to probe decision boundaries and craft evasion attacks. Explore how output formats, query budgets, and transferability influence attack feasibility. Learn to assess evasion claims by clearly defining objectives, access models, and constraints to accurately interpret robustness evaluations.
A prediction API draws a clean boundary around a model. You can send an image to an image-classification API and get back a label, and sometimes you also get a confidence score or a full probability vector.
That boundary still leaks information through outputs. If an attacker can choose inputs and observe outputs repeatedly, the attacker can learn about decision behavior without ever seeing weights, training data, or gradients.
What query access really means at the API boundary
Query access means the attacker can submit chosen inputs to the API and observe the returned outputs. The details matter because the output format changes what the attacker can infer. A label-only response reveals which side of a decision boundary the input fell on, while probabilities reveal how strongly the model prefers alternatives.
APIs also impose limits that shape feasibility. Rate limiting reduces how quickly observations accumulate, logging increases detection risk, and output truncation removes useful detail such as calibrated probabilities.
As you look at the diagram below, ...