Maximum Likelihood Estimation (MLE)

What is MLE?

Maximum likelihood estimation (MLE) is a method used to determine the parameters of a process or model used to generate the observed data. The parameters are found such that they maximize the log-likelihood of the observed data being generated by the process described by the model.

For example, suppose a math teacher wants to grade students based on their performance on a math test. They can use MLE to fit a suitable model, such as normal, binomial, Poisson, and so on, to the test scores and find the most likely values of the mean, standard deviation, proportion, rate, etc., needed to grade students.

To understand better, let’s assume we have $N$ observed data points $D = \{x_1, x_2, ..., x_N\}$ generated by a known distribution $P(x, \beta)$ but parameterized by an unknown parameter $\beta$. We wish to estimate the optimal parameter $\beta^*$ ...