Deep Learning with PyTorch Step-by-Step: Part I - Fundamentals/

...

/

Step 2b - Computing the Loss Surface

Step 2b - Computing the Loss Surface

Learn how the loss can be computed for all possible values in a given range as well as how it can be visually represented.

We'll cover the following...

Loss surface

We have just computed the loss (2.74) corresponding to our randomly initialized parameters (b = 0.49 and w = -0.13). What if we did the same for all possible values of b and w? Well, not all possible values but all combinations of evenly spaced values in a given range like the example shown in the code below:

Python 3.5
# Reminder:
true_b = 1
true_w = 2
# we have to split the ranges in 100 evenly spaced intervals each
b_range = np.linspace(true_b - 3, true_b + 3, 101)
w_range = np.linspace(true_w - 3, true_w + 3, 101)
# meshgrid is a handy function that generates a grid of b and w
# values for all combinations
bs, ws = np.meshgrid(b_range, w_range)
# the shape method below gives the dimensions of the parameters bs and ws
print(bs.shape, ws.shape)

The result of the meshgrid operation was two (101, 101) matrices representing the values of each parameter inside a grid. What does one of these matrices look like?

Let us check this out by displaying one of these in the code below:

Python 3.5
# Printing one of the matrices we got from meshgrid
print(bs)

Sure, we are somewhat cheating here since we know the true values of b and w and can choose the perfect ranges for the parameters. However, it is for educational purposes only so we’ll keep it for now.

Next, we could use those values to compute the corresponding predictions, errors, and losses.

Computing the predictions

Let us start taking a single data point from the training set and computing the predictions for every combination in our grid:

Python 3.5
dummy_x = x_train[0] # taking single data point from training set
dummy_yhat = bs + ws * dummy_x # computing predictions for every combination
print(dummy_yhat.shape)

Thanks to its broadcasting capabilities, Numpy can understand that we want to multiply the same x value by every entry in the ws matrix. This operation resulted in a grid of predictions for that single data ...