Scale It Right

Imagine you’re training a machine learning model that uses income and age to predict loan default. Income might range from $20,000$ to $200,000$, while age falls between $18$ and $90$. If we feed these raw values directly into the model, the algorithm might weigh income more heavily—not because it’s more important, but because its values are numerically larger.

This introduces a silent bias.

Feature scaling transforms numerical values so that each feature contributes more equally during training. It makes learning faster, improves accuracy, and ensures models don't favor features based on their raw scale.

What is feature scaling?

Feature scaling is a preprocessing technique that transforms numeric features so they exist on a comparable scale. This adjustment is crucial when dealing with machine learning algorithms that are sensitive to the magnitude of feature values. Scaling helps in multiple ways:

• Fair comparisons: It ensures that no single feature dominates just because of its scale.

• Faster convergence: In models using gradient descentGradient descent is an optimization algorithm that iteratively adjusts parameters to minimize a function by moving in the direction of the steepest descent. (like linear regression or neural networks), scaled features allow the optimizer to move efficiently toward the minimum, reducing training time.

• Improved performance: Many algorithms perform better and produce more stable results when input features are scaled.

Scaling approaches

The following two common scaling approaches are commonly used to bring features onto a similar scale. The choice between them depends on the data distribution and the requirements of the machine learning algorithm.

1. Normalization (Min-max scaling): Rescales values into a fixed range—typically $[0, 1]$ ...