Hierarchical and Density-Based Clustering
Explore hierarchical and density-based clustering techniques to group incident reports, detect noise points, and choose the best method based on cluster shape and operational needs. Understand how to interpret dendrograms, tune parameters, and handle non-spherical clusters while preserving actionable categories.
Incident reports can form two curved groups in a two-dimensional feature space while a few rare reports sit far from both groups. K-means must assign every report to a centroid, so it can cut one curved group into pieces or attach an unusual report to an ordinary segment.
Non-spherical clusters are groups whose points form shapes that are not compact and roughly round around a centroid. A crescent-shaped group is non-spherical because points can be close along the curve while remaining far from one shared center. Outliers, also called noise points in density-based clustering, are isolated observations that do not belong to a sufficiently supported group.
For incident triage, forcing an isolated report into a segment hides the condition that matters. We need methods that can show a hierarchy of possible groupings or leave unusual reports unassigned.
Merge points from the bottom up
Agglomerative hierarchical clustering starts with each incident report in its own cluster. It then finds the closest pair of clusters and merges them. A linkage rule defines how the algorithm measures the distance between two clusters. The algorithm repeats the merge until all reports belong to one large cluster.
The result is a dendrogram, a branching diagram that records the sequence of merges. Its vertical axis represents merge distance. A low merge joins clusters that were close together. A ...