Underfitting and Overfitting

Model complexity and error

Crafting valuable machine learning models is an iterative process. As a machine learning practitioner, it’s common to perform multiple iterations of training machine learning models based on the following:

• Changes and improvements to the training set used by the machine learning algorithm.

• Different values and combinations of hyperparameters to tune the model.

• Experimenting with different algorithms (e.g., decision trees vs. random forests).

All of these have one thing in common—they directly impact the complexity of the machine learning model.

For any given problem, a level of model complexity produces the optimal predictions (i.e., the predictions with the lowest error). Conceptually, we can graph this relationship as follows: