k-Nearest Neighbors and Naive Bayes
Explore the mechanisms behind k-Nearest Neighbors and Naive Bayes classifiers in this lesson. Understand how k-NN uses similarity and distance metrics with feature scaling for predictions, while Naive Bayes applies probabilistic evidence under conditional independence assumptions. Learn to select and interpret these models for classification tasks, including their strengths and practical considerations for support ticket prioritization.
For support-ticket prioritization, a simple classifier can decide whether a ticket is urgent or not_urgent before a human reviews it. Suppose each ticket has waiting_minutes and affected_users. Two baseline classifiers can use the same records but make predictions for different reasons. k-nearest neighbors relies on similarity between tickets, while Naive Bayes combines evidence from ticket features into a probability.
k-nearest neighbors (k-NN) predicts a class from the labels of the most similar stored training records. A neighbor is one stored record close to the new record. The value k is the number of nearby records included in the decision. If three of the five nearest tickets were urgent, k-NN predicts urgent for the new ticket.
To decide what counts as close, k-NN needs a distance metric, a rule that turns differences between feature values into a numeric distance. For two numeric features, Euclidean distance is a common default. It measures straight-line distance between points. Other distance metrics exist, but the essential requirement stays the same. The feature space must make similarity meaningful.
k-NN can also estimate an urgent probability as the fraction of selected neighbors labeled urgent. With k=5 and four urgent neighbors, the neighbor-based probability is