Multiclass Formulation

Multiclass extension

The logistic regression model offers two significant advantages: the ability to learn probabilities and the natural extension to handle multiple classes. Let’s explore how we can extend a binary classification model to a multiclassification model with $c$ classes. This technique is also called one-vs-all (one-vs-rest).

Algorithm

For each class $j$ in the dataset:

1. Set the labels of class $j$ to 1, indicating positive instances.

2. Set the labels of all other classes to 0, representing negative instances.

3. Apply logistic regression on the modified dataset, treating it as a binary classification problem with class $j$ as the positive class (1) and all other classes as the negative class (0). Save the corresponding model parameters $\boldsymbol{\bold w_j}$.

4. Predict the probability $\hat{y}_{tj} = \sigma(\bold w^T_j\phi(\bold x_t))$ ...