Search⌘ K
AI Features
Home
Courses
Data Science Bootcamp

Support Vector Machines

Explore how support vector machines (SVMs) classify data by maximizing the margin between classes and using support vectors to define the boundary. Understand soft margins and the role of the regularization parameter C in handling overlapping data. Discover how kernels, such as the radial basis function, enable SVMs to model complex nonlinear boundaries. This lesson teaches you to interpret SVM decision functions and the impact of parameter tuning on model behavior.

We'll cover the following...

A suspicious-login classifier may separate clearly normal logins from clearly suspicious ones, yet the cases near the dividing line deserve the most attention. A decision boundary is the line or surface where a classifier changes from one class prediction to the other. For two login signals, such as failed_attempts and new_device_score, the boundary is a line on a two-dimensional plot.

A boundary can separate the same training logins in several ways. A margin is the distance from the boundary to the nearest training points from either class. A support vector machine (SVM) chooses a separating boundary with the largest possible margin. The informal goal is simple: separate suspicious and normal logins while leaving as much space as possible between the boundary and the closest examples.

A larger margin does not mean every login will be classified correctly. It means the model explicitly bases its boundary on the difficult cases near that boundary, rather than treating every training point as equally decisive.

The points that hold up the boundary

The nearest logins determine the margin, so an SVM records those decisive points as support vectors. A support vector is a training example on the margin or close enough to it that it affects the boundary. Training examples far from the boundary usually do not affect where a linear SVM places its line.

The following example fits a linear SVM to small two-signal login data. It reports the positions of the support vectors and scores from decision_function. A positive score lies on the side predicted as suspicious, a ...

Python
import numpy as np
from sklearn.svm import SVC
def main() -> None:
    # Small 2D login dataset: (failed_attempts, new_device_score)
    X = np.array(
        [
            [0, 0.10],
            [1, 0.20],
            [0, 0.30],
            [2, 0.25],
            [3, 0.40],
            [2, 0.60],
            [4, 0.65],
            [3, 0.80],
            [5, 0.90],
            [6, 0.85],
        ],
        dtype=float,
    )
    # 0 = normal, 1 = suspicious
    y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1], dtype=int)
    model = SVC(kernel="linear", C=1.0)
    model.fit(X, y)
    print("Support vector indices (support_):")
    print(model.support_)
    print()
    print("Support vectors (support_vectors_):")
    print(model.support_vectors_)
    print()
    print("Decision function scores for each training sample (decision_function(X)):")
    scores = model.decision_function(X)
    for i, score in enumerate(scores):
        print(f"row={i:2d}  x={X[i].tolist()}  y={y[i]}  score={score:.6f}")
if __name__ == "__main__":
    main()
...