Cross Entropy
Understand the cross-entropy loss function and how it measures the difference between predicted and actual labels in neural networks. Learn why it pairs well with softmax activation and how it aids gradient descent training despite potential local minima challenges.
We'll cover the following...
What is the cross-entropy loss?
So far, we have used the log loss formula for our binary classifiers. We even used the log loss when we bundled ten binary classifiers in a multiclass classifier (in The Final Challenge). In that case, we added together with the losses of the ten classifiers to get a total loss.
While the log loss served us well so far, it’s time to switch to a simpler formula, one that’s specific to multiclass classifiers. It’s called the