Probabilistic Classification

Learn about logistic regression and the MLE of the Bernoulli model.

We have already discussed some classification methods such as support vector machines and decision trees. We’ll now discuss how classification can be seen as an important special case of probabilistic regression. In classification, features are mapped to a finite number of possible categories. We’ll now discuss binary classification again where the two target classes are usually denoted as 00 and 11. Later, we can easily generalize the ideas to more classes. We’ll start by describing a binary random variable.

Maximum likelihood estimation (MLE) of the Bernoulli model

An important probabilistic model of a binary random variable is the Bernoulli model. In this model, a random number takes the value of 11 with probability ϕ\phi and the value 00 with probability 11 ϕ\phi (the probability of being either of the two choices has to be 11). This can be nicely combined into the following formula:

Get hands-on with 1200+ tech skills courses.