Recap
Explore the core concepts of gradient descent in this recap lesson. Understand how parameters are updated using gradients and learning rates, the importance of feature scaling, and the differences between batch types. Gain practical insights into loss surfaces and update steps, preparing you to train models effectively in PyTorch.
We'll cover the following...
General overview
This finishes our journey through the inner workings of gradient descent. By now, we hope you are able to develop better intuition about the many different aspects involved in the process.
In time and with practice, you will observe the behaviors described here in your own models. Make sure to try plenty of different combinations: mini-batch sizes, learning rates, etc. This way, not only your models will learn but so will you!
This is a (not so) short recap of everything we covered in this chapter:
-
Defining a simple linear regression model.
-
Generating synthetic data for it.
-
Performing a train-validation split on our dataset.
-
Randomly initializing the parameters of our ...