Finding the Optimal Number of Clusters
Explore methods for determining the optimal number of clusters in KMeans clustering. Learn to apply the elbow technique by analyzing inertia values to balance model precision and complexity. Understand how multiple dimensions affect clustering and the trade-offs in cluster selection for practical machine learning applications.
We'll cover the following...
We will now see the options we have in choosing the optimal number of clusters and what that entails, but let’s first take a look at the following screenshot to visualize how things progress from having one cluster to eight clusters:
We can see the full spectrum of possible clusters and how they relate to data points. At the end, when we specified “8,” we got the perfect solution in which every data point is a cluster center.
In reality, we might not want to go for the full solution, for two main reasons.
- Firstly, it is probably going to be prohibitive from a cost perspective. Imagine making 1,000 T-shirts with a few hundred sizes.
- Secondly, in practical situations, it usually wouldn’t add much value to add more clusters after a certain fit has been achieved.
Using our T-shirt example, imagine if we have two people with sizes ...