Understanding the principles

Clustering: Similarity, Cluster Shape, and Use Case Jointly Determine Grouping

From manual K-means assignment–update computation to hierarchical clustering, DBSCAN, mixture models, choosing k, stability, and business interpretation.

Core idea Clustering has no task-independent ground truth. K-means, hierarchical clustering, DBSCAN, and mixture models treat spherical distance, linkage rules, local density, or probability distribution as “clusters,” respectively. Result quality depends first on representation, scale, cluster shape, and use case, and only second on the algorithm name; every cluster should be validated through stability, external evidence, and action risk.

After reading, you should be able to:Manually compute one K-means assignment and update; choose algorithms by cluster shape; explain k, ε, and linkage rules; validate stability and avoid over-naming.

1Clustering first answers “what does similarity mean”Definition

Why can the same batch of users result in different groupings by consumption, category, or region?

ClusteringClustering is the process of grouping samples according to pre-selected similarity rules when no human-provided category labels are available. It addresses “whether reusable structure exists in the data,” rather than discovering a single true identity for samples. The input consists of each object’s feature representation, a distance or similarity measure, and algorithm parameters; the output is a cluster label, and may also include centers, a hierarchical tree, noise markers, or soft assignment probabilities.

Feature selection, scaling, and distance together define proximity. Euclidean distance emphasizes absolute differences, cosine distance emphasizes direction, and edit distance emphasizes sequence transformations. The algorithm searches for structure that is relatively similar within groups and relatively different between groups according to this definition. Cluster labels themselves carry no inherent ordering or natural meaning; you must inspect representative samples and feature distributions before naming them.

Boundary:If the representation omits information needed by the task, or treats mixed factors such as device or region as the main differences, clustering will consistently produce a useless or even harmful grouping; the algorithm cannot choose meaningful semantics for you.

Running example:Five two-dimensional points A=(1,1), B=(1,2), C=(2,1), D=(8,8), E=(9,8). In this example the input consists of five coordinates, set the number of clusters k=2, initial centers μ₁=A, μ₂=D; the output will be the cluster to which each point belongs and the two updated centers.

2K-means Minimizes Within-Cluster Sum of SquaresObjective

Why must the center be the mean of the samples assigned to it?

K-means solves the problem of “how to summarize numerical vectors using k centers.” The inputs are n vectors xᵢ on the same scale and a pre-specified number of clusters k; the outputs are a cluster index cᵢ for each sample, the center μₖ of each cluster, and an objective value J that measures within-cluster compactness.

J=Σ_i=1…n‖xᵢ−μ_cᵢ‖²

Here i is the sample index, xᵢ is the i-th vector, cᵢ is its current cluster assignment, μ_cᵢ is the center of that cluster, ‖·‖² is the squared Euclidean distance, and J is the sum of squared distances from all samples to their respective centers. With the centers fixed, choosing the nearest center for each point independently reduces J; with the assignments fixed, taking the derivative with respect to the center and setting it to zero gives the mean of the cluster as the optimal center. The two steps alternate, J does not increase, and the procedure eventually stops.

A smaller J only indicates that the samples are closer to their centers at the current scale, and it cannot be directly interpreted as better business groupings. The algorithm only guarantees reaching a local optimum, not a global optimum; different initializations can lead to different solutions, and non-spherical clusters or unscaled features can also distort the result.

Point	Squared distance to μ₁=(1,1)	to μ₂=(8,8)	Assignment
A	0	98	1
B	1	85	1
C	1	85	1
D	98	0	2
E	113	1	2

3Full manual calculation: recalculate the two centers after assignmentStep-by-step calculation

Where will the first update move the centers, and how much will the objective decrease?

This section applies the previous section's “assign–update” rule to five given coordinates, with the goal of seeing the input and output of one iteration. The assignment step inputs the old centers, compares squared distances in the table, and outputs cluster 1={A,B,C}, cluster 2={D,E}; the update step then averages each cluster's coordinates dimension by dimension.

Cluster 1's new center μ₁=((1+1+2)/3,(1+2+1)/3)=(4/3,4/3); cluster 2's new center μ₂=((8+9)/2,(8+8)/2)=(8.5,8).

J_old=0+1+1+0+1=3; J_new=4/3+1/2=11/6≈1.833

J_old is the within-cluster sum of squares before the update, J_new is the within-cluster sum of squares after the update; from 3 down to about 1.833, showing that the new centers are more compact for the current grouping. After reassignment the assignments do not change, so the algorithm converges in this example, but this does not prove that a global optimum or the true classes have been obtained. This nice result comes from two compact spherical clusters; if the points are arranged as a crescent, have different densities, or contain outliers, the mean and Euclidean distance will be distorted.

4Original figure: The same set of points observed under different cluster assumptionsVisualization

What do spherical, density-connected, and hierarchical partitioning see respectively?

Scroll horizontally to view the full diagram on small screens.

Figure 1 K-means partitions into Voronoi regions using centroids; DBSCAN connects density regions; hierarchical methods preserve multi-granularity structure. No single shape assumption fits all data.

5K-means prefers spherical, similar variance, and similar sizeBoundary

Why do elongated clusters, different densities, and outliers mislead the centroid?

Squared Euclidean distance penalizes distant points heavily; outliers can significantly drag the mean. Nearest-centroid boundaries are linear and struggle to represent crescents and rings. Large clusters may be split, and small clusters may be absorbed. Standardization only addresses scale and does not fix shape assumptions.

Problem	Symptom	Candidate method
Outliers	Center is dragged away	k-medoids, robust handling
Crescent/ring	Cut apart by straight lines	DBSCAN, spectral clustering
Soft boundary	Hard assignment at critical points	Gaussian mixture
Multiple granularities	k hard to fix	Hierarchical clustering

6DBSCAN Connects Arbitrary Shapes Using Local DensityDensity Method

How do ε and minPts together define core points?

DBSCAN is density clustering: it solves the problem that curved clusters and noise points cannot be well represented by centroids. The input is samples, a distance function, a neighborhood radius ε, and a minimum number of points minPts; the output is several density-connected clusters, boundary points, and noise labels.

N_ε(x)={y:d(x,y)≤ε}; |N_ε(x)|≥minPts ⇒ core point

N_ε(x) denotes the ε-neighborhood of point x, y is a candidate neighbor, d(x,y) is the distance between two points, |N_ε(x)| is the number of neighborhood points. When the number of points reaches minPts, x is a core point; if the neighborhoods of core points are mutually reachable, they are assigned to the same cluster, boundary points can be absorbed, and sparse points are labeled as noise. Noise is not “wrong data”; it is just points that are not included in dense regions at the current density scale.

It does not require presetting k and can track curved shapes; however, when different densities coexist, a single ε is hard to balance, and the convergence of distances in high dimensions can also make the neighborhood lose discriminative power. Plotting the k-distance curve first is only a heuristic, not an automatic ground truth; you should perform ε×minPts sensitivity and resampling stability.

7Hierarchical clustering encodes “how to merge” into the linkage ruledendrogram

Why do single, complete, and average linkage produce different trees?

Linkage	Inter-cluster distance	Typical tendency
single	nearest pair of points	Can follow curved shapes, but prone to chaining
complete	farthest pair of points	Compact, sensitive to outliers
average	average point pair	Compromise
Ward	Within-cluster variance increment	Approximately spherical

A dendrogram retains multiple levels of granularity, but once a greedy merge is made, it is usually not undone; early errors propagate. The height at which the tree is cut should be chosen based on stable intervals and the intended use, not by finding the most visually pleasing horizontal line.

8Gaussian mixture models turn hard assignments into posterior probabilitiesSoft clustering

Why can a user located between two clusters have 60%/40% membership?

p(x)=Σ_kπₖ𝓝(x|μₖ,Σₖ), rᵢₖ=P(zᵢ=k|xᵢ)

EM's E-step computes the responsibilities r, and the M-step uses soft weights to update π, μ, and Σ. Covariance allows elliptical clusters, and soft assignment expresses boundary uncertainty; however, components still rely on the Gaussian assumption, and the likelihood can become numerically unstable due to covariance collapse, requiring regularization.

Components are not natural populations.The probabilistic model only indicates which component is more likely to have generated the data under the current parameters; it does not assign social or causal identity.

9Choosing k is not about letting a single curve make the decision for youModel Selection

Why does inertia always decrease as k increases?

Choosing k addresses “how fine-grained a grouping remains useful.” The input is clustering results for multiple candidate k values, internal metrics, stability, and business constraints; the output is a primary granularity and the alternative granularities that need to be reported. The approach is to retrain each candidate and then compare metrics and resampling consistency.

When k=n, each point is its own cluster, and K-means inertia can be 0, so you cannot choose the minimum.Silhouette coefficientIt compares a sample’s average distance to points in its own cluster with its average distance to points in the nearest other cluster: close to 1 indicates both compact and well-separated, close to 0 indicates boundary, and a negative value suggests possible misassignment. Elbow, silhouette, Gap, and BIC/AIC all carry assumptions and may disagree. You should look for a granularity where results are stable, interpretable, and support action, and report alternative k values; internal geometric scores cannot replace real business outcomes.

Evidence	Use	Limitations
Elbow	Marginal benefit	Subjective inflection point
Silhouette	Compactness and separation	Favors convex clusters
BIC/AIC	Probabilistic model complexity	Depends on the distribution family
Business constraints	Actionable granularity	Requires external validation

10Stability and semantic validation matter more than a single best scoreAcceptance

If changing the random seed or month rearranges the clusters, how do you judge whether they can still be used?

Use multiple initializations, comparing the objective value and the sample co-clustering matrix.
Retrain with bootstrap/time slices and compare using ARI/NMI or optimal matching.
Inspect representative samples, feature distributions, and boundary points.
Check for confounds such as device, region, and missingness patterns.
If it drives operations, run controlled experiments to measure real gains and harm.

Cluster labels are permutable, so you cannot directly compare the name “Cluster 1”; first match by sample overlap or optimal centroid matching, then judge splits, merges, and drift.

11Common Misconceptions and Learning PathMisconceptions and Dependencies

Clusters are model artifacts; naming must occur after validation.

Misconception	More Accurate Understanding
Clustering can find a single natural category	Grouping depends on representation, scale, and assumptions
K-means always finds the global optimum	Alternating optimization only guarantees local convergence
DBSCAN requires no hyperparameters	ε and minPts determine the density scale
The highest silhouette means the best k	Internal geometry does not equal business value
Cluster labels have fixed semantics	After retraining, labels can be arbitrarily permuted

Level	Dependencies and Extensions
Prerequisites	Distance, mean, variance, probability
Core on this page	K-means, density, hierarchical, soft assignment
Diagnostics	Dimensionality reduction, curse of dimensionality, anomaly detection
Governance	Fairness, drift, human review

12Online clustering must also handle drift and cold startProduction Boundary

When a new user arrives, should they be directly assigned to an existing cluster, or should the model be retrained immediately?

A stable production system typically first uses frozen centroids or a trained model to assign new samples, then monitors distance, cluster size, and unassigned rate over time windows; it retrains only when evidence indicates structural change. After retraining, you need to match old and new clusters, review splits and merges, and gradually migrate downstream policies. Frequent retraining makes business labels unstable; never retraining forces drift into an outdated structure.

13Connect the causal chainSynthesis

How does this concept connect from a problem all the way to verifiable practice?

Define purpose, representation, and distance
Choose an algorithm that matches the cluster shape
Use multiple initializations and tune granularity parameters
Use resampling matching to check stability
Explain with samples and domain knowledge
Decide adoption based on downstream gains and fairness risks

14Misconceptions and Self-TestSelf-Test

Can you explain its mechanism, boundaries, and validation methods without memorizing terminology?

What are the two centers after the first update?
In this example, what does J decrease from and to?
How is a core point defined in DBSCAN?
What is a typical risk of single linkage?
Why can't you directly compare "cluster 1" from two different runs?
Assume "Clustering: Similarity, Cluster Shape, and Use Case Jointly Determine Grouping" performs normally on offline examples, but core results decline after going live. How would you locate the problem by input, internal transformation, output feedback, and applicable boundaries?

Reference Answers

μ₁=(4/3,4/3), μ₂=(8.5,8).
It decreases from 3 to 11/6≈1.833.
Within its ε-neighborhood, there are at least minPts points.
A few bridging points cause chaining.
Cluster labels can be arbitrarily permuted; you must first match by sample overlap or centers.
First save the same failing sample and environment, and confirm that inputs, permissions, and preconditions have not drifted. Then record key intermediate states and check whether the mechanism completed the transformation as described on this page. Next, compare the raw output with independent metrics and manual final review. Finally, retest using boundary examples and controlled experiments. Only after locating the first stage that deviates from expectations can you determine whether to modify the data, mechanism, evaluation, or usage boundaries.

Sources and adaptation notes

k-means++: The Advantages of Careful Seeding: initialization and approximation guarantees
DBSCAN: density clustering and noise
The Elements of Statistical Learning: clustering and mixture models
Finding Groups in Data: cluster analysis and validation

Accessed: 2026-07-22