Occam’s Razor

Get introduced to Occam’s Razor and compare bias-variance trade-off.

What is Occam’s Razor?

Occam’s Razor is credited to the English philosopher William of Ockham. It states that out of two or more competing theories, the simpler theory is to be preferred.

In this lesson, we’ll see this theory from the lens of machine learning. Occam’s Razor is intuitive and is easily understood when applied to real-life concepts. Similarly, we can say that a simple model should be preferred in machine learning.

What is a simple model?

A machine learning model is a parametric function, and the number of parameters is a quantification of its complexity. The number of parameters is directly proportional to the complexity of the model.

For example, let’s take a task TT for which we have two models M1M_1 and M2M_2. This is shown in the figure below:

Get hands-on with 1200+ tech skills courses.