Introduction to Logistic Regression

Logistic Regression

Logistic regression is a discriminative model widely used for classification tasks. The term “logistic” in logistic regression refers to the utilization of the logistic function. Consider a binary classification problem: the target variable $y_i$ can take on values in $\{0, 1\}$. One way to model the probability distribution of the target label $y_i$ being equal to 1 given the feature vector $\phi(\bold{x}_i)$ is by employing a logistic function defined as:

$$\hat{y}_i=p(y_i=1|\phi(\bold{x}_i))=\frac{1}{1+e^{-\bold w^T \phi(\bold{x}_i)}}$$

The complementary probability $p(y_i=0|\phi(\bold x_i))$ can be obtained as:

$$p(y_i=0|\phi(\bold{x}_i))= 1-p(y_i=1|\phi(\bold{x}_i))$$

Sigmoid function

A logistic function is also known as sigmoid, typically denoted by $\sigma(z)$.

$$\sigma(z)=\frac{1}{1+e^{-z}}$$