Decision-Boundary Intuition
Explore the concept of decision boundaries in high-dimensional input spaces and understand how small input changes can cause misclassification during inference. Learn to distinguish adversarial examples from natural errors, comprehend attacker objectives, and relate model robustness to the margin around decision boundaries.
We'll cover the following...
You send an image to a deployed image-classification model through a prediction API, then you send a second image that looks the same to you, and the top label flips to a different class. Nothing about the server changed between calls, and the two inputs differ only in tiny pixel values. That behavior is the starting point for adversarial examples.
Several explanations can sound plausible at first:
Randomness: Can make outputs vary if the service injects noise or uses nondeterministic hardware paths.
Pipeline bugs: Can scramble preprocessing so two similar inputs map to very different internal arrays.
Distribution shift: Can also do it if the second image actually contains a subtle feature the model relies on more than you do.
Inference-time integrity is narrower and more specific. The attacker does not retrain the model, does not edit weights, and does not need access to your codebase. The attacker only chooses the input to your prediction ...