Creating Parameters
Explore how to create tensors that serve as trainable parameters in PyTorch by enabling gradient computation with requires_grad. Understand different approaches to assign tensors to devices, ensuring GPU support and reproducibility. This lesson guides you through practical steps and best practices for efficient parameter creation in model development.
We'll cover the following...
Tensors requiring gradients
What distinguishes a tensor used for training data from a tensor used as a trainable parameter/weight?
The latter requires the computation of its gradients, so we can update their values (the parameters’ values, that is). That is what the requires_grad=True argument is good for. It tells PyTorch to compute gradients for us.
A tensor for a learnable parameter requires a gradient!
You may be tempted to create a simple tensor for a parameter and send it to your chosen device later on; this was similarly done with our data, right? Not so fast.
...
In the next few segments, you will be presented with four chunks of code that show different attempts at creating parameters.
The first three attempts are shown to build up a solution. The first one only ...