Logistic Regression

Logistic Regression is a famous classification algorithm, and it is used extensively in the field. You'll learn more about it here.


Classification problems involve assigning labels or classes to instances. For example, an email can be spam or ham (not Spam). Logistic Regression is the basic algorithm used for the Binary Classification problems. It can be extended for Multi-class and Multi-label Classification problems. In this lesson, we will cover the mathematical working of Logistic Regression and how it can be used in Scikit Learn for real time problems.

Logistic Regression

Logistic Regression allows us to estimate the probability, pp, of class membership of a given instance. The probability will range between 0 and 1. Logistic regression is also known in the literature as Logit regression, Maximum-Entropy Classification (MaxEnt), or the Log-Linear classifier.
We saw that Multivariate Linear Regression comes up with the following equation in higher dimensions.

y^=w0x0+w1x1+w2x2+w3x3+w4x4\hat{y} = w_0 * x_0 + w_1 * x_1 + w_2 * x_2 + w_3 * x_3 + w_4 * x_4wnxnw_n * x_n

Here x0x_0 = 1

y^=wTx\hat{y} = w^T * x

In Logistic Regression, we pass the calculated predicted value y^\hat{y} through a Logistic or Sigmoid function g(z)g(z).

y^Logistic\hat{y}_{Logistic} = g(y^)g(\hat{y})


g(z)g(z) = 11+ez\frac{1}{1+e^{-z}}

g(y^)g(\hat{y}) = 11+ey^\frac{1}{1+e^{-\hat{y}}} = y^Logistic\hat{y}_{Logistic}

Get hands-on with 1200+ tech skills courses.