Goodness of Fit and Information Criteria
Explore how to assess and compare time series models by understanding goodness of fit metrics such as the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). This lesson helps you evaluate model accuracy while avoiding overfitting, using practical examples with Python's statsmodels package to identify the best fitting model for your data.
We'll cover the following...
Motivation
When developing a time series model, we want to ensure that the model fits the training data. The extent to which it fits the data is what we call the goodness of fit of the model. Calculating a goodness-of-fit metric also allows us to compare and rank models among an array of possible specifications. There are many ways to test whether or not a model fits its training data in a satisfactory way. In this lesson, we will study two famous goodness-of-fit metrics: the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).
In this context, the term “information” refers to the notion that these criteria involve a trade-off between the complexity of the model and the amount of information gained by fitting it to the data. In other words, models with more parameters tend to fit their training data better but at the risk of overfitting it. Therefore, both the AIC and the BIC try to strike a balance between explaining as much variation in the training data as possible and not having a lot of parameters.
Akaike information criterion
The Akaike information criterion (AIC) is defined as follows:
In the formula above,