Scaling the Dataset

Overview of learning rate results

The conclusion that we drew from looking at the results of the different learning rates was that it is best if all the curves are equally steep, so the learning rate is closer to optimal for all of them!

Achieving equally steep curves

How do we then achieve equally steep curves? The short answer: you have to “correctly” scale your dataset. Let us now go into depth about how scaling your dataset helps to achieve equally steep curves.

“Bad” feature

First, let us take a look at a slightly modified example, which we would be calling the “bad” dataset:

• Over here, we multiplied our feature (x) by 10, so it is in the range [0, 10] now and renamed to bad_x.

• But since we do not want the labels (y) to change, we also divided the original true_w parameter by 10 and renamed it bad_w. this way, both bad_w * bad_x and w * x yields the same results.