Computation Graphs

That automatic gradient calculation in the last section seems magical, but of course, it isn’t.

It is worth understanding just a little bit about how the automatic gradient calculation is done because this knowledge will help us later when we build larger networks.

Simple computation graph

Take a look at the following very simple network. It’s not a neural network, just a sequence of calculations.