Binary Cross-Entropy Loss in PyTorch
Explore how to implement and use binary cross-entropy loss functions in PyTorch for binary classification tasks. Learn the differences between BCELoss and BCEWithLogitsLoss, when to apply each, and how the order of inputs affects loss calculation. This lesson helps you build stable and accurate binary classification models by correctly combining model outputs with appropriate loss functions.
We'll cover the following...
BCELoss
Sure enough, PyTorch implements the binary cross-entropy loss, [nn.BCELoss]. Just like its regression counterpart, MSELoss (introduced in the chapter, A Simple Regression Problem), it is a higher-order function that returns the actual loss function.
The BCELoss higher-order function takes two optional arguments (the others are deprecated, and you can safely ignore them):
-
reduction: It takes eithermean,sum, ornone. The defaultmeancorresponds to our equation 6.15 in the previous lesson. As expected,sumwill return the sum of the errors instead of the average. The last option,none, corresponds to the unreduced form; that is, it returns the full array of errors. -
weight: The default isnone. Meaning, every data point has equal weight. If informed, it needs to be a tensor with a size that equals the number of elements in a mini-batch, representing the weights assigned to each element in the batch. In other words, this argument allows you to ...