Search⌘ K
AI Features
Home
Courses
An Introductory Guide to Data Science and Machine Learning

Logistic Regression

Explore the theory and application of logistic regression for binary and multi-class classification. Understand how probabilities are estimated, cost functions are minimized using gradient descent, and how regularization improves model performance. Gain practical skills using scikit-learn to train and evaluate logistic regression models on real datasets.

We'll cover the following...

Classification

Classification problems involve assigning labels or classes to instances, such as determining whether an email is spam or not spam. Logistic regression is the basic algorithm used for binary classification problems. It can be extended for multi-class and multi-label classification problems. In this lesson, we will cover the mathematical working of logistic regression and how it can be used in scikit-learn for real-time problems.

Logistic regression

Logistic regression allows us to estimate the probability, pp, of class membership of a given instance. The probability will range between 0 and 1. Logistic regression is also known in the literature as Logit regression, Maximum-Entropy Classification (MaxEnt), or the log-linear classifier.
We saw that multivariate linear regression comes up with the following equation in higher dimensions.

y^=w0∗x0+w1∗x1+w2∗x2+w3∗x3+w4∗x4\hat{y} = w_0 * x_0 + w_1 * x_1 + w_2 * x_2 + w_3 * x_3 + w_4 * x_4 … wn∗xnw_n * x_n

Here x0x_0 = 1

y^=wT∗x\hat{y} = w^T * x

In logistic regression, we pass the calculated predicted value y^\hat{y} through a logistic or sigmoid function g(z)g(z).

y^Logistic\hat{y}_{Logistic} = g(y^)g(\hat{y})

Where,

g(z)g(z) = 11+e−z\frac{1}{1+e^{-z}}

g(y^)g(\hat{y}) = 11+e−y^\frac{1}{1+e^{-\hat{y}}} = y^Logistic\hat{y}_{Logistic} ...