H. Sherry Zhang Department of Statistics and Data Sciences The University of Texas at Austin
Fall 2026
Learning objectives
Understand the K-means and hierarchical clustering algorithms on a high level
Prepare data for clustering: remove missing values (na.omit()), scale variables (scale()), and select numeric variables
Perform K-means clustering in R:
choose K with domain knowledge, the elbow method, or the silhouette method (factoextra::fviz_nbclust())
fit with kmeans(centers = , nstart = ) and use set.seed() for reproducible results
add cluster labels back to the data and interpret clusters with summaries and plots
(Appendix) Perform hierarchical clustering in R: 1) compute distances with dist() and fit with hclust(), 2) compare linkage methods (single, complete, average, Ward’s) using dendrograms, 3) cut the tree into K clusters with cutree() and compare the results with K-means
Cluster analysis in life
Imagine you’re a supermarket manager deciding how to arrange these sports equipment onto two shelves:
dumbbell
barbell
hiking boots
golf club
tennis racket
resistance band
One solution:
Shelf 1, gym equipment: dumbbell, barbell, resistance band
There’s no single “correct” answer: it depends on how we define similar.
Cluster analysis
Clustering groups observations so that items in the same cluster are similar to each other and different from items in other clusters.
It is unsupervised: there is no outcome variable or label to predict, only structure to find.
Two common approaches:
K-means clustering: we choose the number of clusters, K, in advance (K = 2 shelves above).
Hierarchical clustering: we don’t fix K in advance. The algorithm builds a tree of nested clusters (a dendrogram), and we choose K afterwards by cutting the tree.
Example: Penguins data
as_tibble(penguins)
# A tibble: 344 × 8
species island bill_len bill_dep flipper_len body_mass sex year
<fct> <fct> <dbl> <dbl> <int> <int> <fct> <int>
1 Adelie Torgersen 39.1 18.7 181 3750 male 2007
2 Adelie Torgersen 39.5 17.4 186 3800 female 2007
3 Adelie Torgersen 40.3 18 195 3250 female 2007
4 Adelie Torgersen NA NA NA NA <NA> 2007
5 Adelie Torgersen 36.7 19.3 193 3450 female 2007
6 Adelie Torgersen 39.3 20.6 190 3650 male 2007
7 Adelie Torgersen 38.9 17.8 181 3625 female 2007
8 Adelie Torgersen 39.2 19.6 195 4675 male 2007
9 Adelie Torgersen 34.1 18.1 193 3475 <NA> 2007
10 Adelie Torgersen 42 20.2 190 4250 <NA> 2007
# ℹ 334 more rows
Make sure you’re using the base R penguins dataset, not the one from the palmerpenguins package.
K-means with the penguins data
Question: using only body measurements, can we find groups of penguins that look like the different species?
This is an unsupervised problem: we set species aside, cluster on the measurements, and then check whether the clusters line up with species.
The kind of structure we might find (not necessarily true):
A penguin with a small bill depth is likely to be a Gentoo.
A penguin with a short bill AND a deep bill is likely to be an Adelie.
What’s the issue with variables being in different scales?
K-means pre-processing: variable selection
Some variables in the dataset are categorical (island, sex). Should we include them?
No, K-means does not allow categorical variables. It computes cluster means and Euclidean distances, which only make sense for numbers: what is the average of “Biscoe” and “Dream”? If you try, R gives an error:
kmeans(penguins_clean[, c("bill_len", "island")], centers =2)#> Error in do_one(nmeth) : NA/NaN/Inf in foreign function call (arg 1)
Recoding categories as numbers (Biscoe = 1, Dream = 2, Torgersen = 3) runs, but is misleading: it treats Torgersen as “twice as far” from Biscoe as Dream is.
Gentoo penguins are only found on Biscoe, Chinstrap only on Dream, and Torgersen only has Adelie.
Extensions for mixed data types (optional)
Extensions that handle a mix of numeric and categorical variables:
Gower distance (cluster::daisy(metric = "gower")): a distance that works with mixed variable types. Pair it with K-medoids (cluster::pam()), a K-means variant where each cluster center is an actual observation (the medoid), so it only needs distances, not means.
The K-means algorithm requires the user to specify the number of clusters, K, in advance. How to choose?
Based on domain knowledge
Elbow method: plot the total within-cluster variation against the number of clusters K, and look for an “elbow” in the plot.
Silhouette method: measure how far each observation is from the observations in its own cluster compared to those in the nearest cluster. Average silhouette width can be computed for different values of K, and the value of K that maximizes the average silhouette width is chosen.
Different methods may give different suggestions for the optimal number of clusters.
Use factoextra package to visualize the results of these methods: fviz_nbclust(DATA, kmeans, method = "...") function.
Elbow (method = "wss"): the total within-cluster variation always drops as K grows; look for where the drop levels off (around K = 2 or 3 here).
Silhouette: starts at K = 2, since with one cluster there is no “nearest other cluster” to compare to. It suggests K = 2, even though there are 3 species: Adelie and Chinstrap overlap a lot on these measurements.
K-means is random: nstart
K-means looks for the clusters that minimize the total within-cluster variation (the sum of squared Euclidean distances from each point to its cluster center).
It starts from randomly chosen centers, then repeats: assign each point to its nearest center, move each center to the mean of its points.
It stops when nothing changes, but this may only be a local optimum: an unlucky start can get stuck on worse clusters.
nstart runs the algorithm nstart times, each from a different random start, and keeps the run with the smallest total within-cluster variation.
K-means is random: set.seed()
set.seed() makes the result exactly reproducible. Even with nstart = 20, the same clusters can get different labels in different runs:
Single linkage: distance between two clusters is defined as the minimum distance between any single member of one cluster and any single member of the other cluster.
Complete linkage: distance between two clusters is defined as the maximum distance between any single member of one cluster and any single member of the other cluster.
Average linkage: distance between two clusters is defined as the average distance between all pairs of members from the two clusters.
Ward’s linkage: distance between two clusters is defined based on the increase in the total within-cluster variance that would result from merging the two clusters.
Selection:
Single linkage are most popular among statisticians.
Average and complete linkage are generally preferred over single linkage, as they tend to yield more balanced dendrograms
Hierarchical clustering in a smaller penguins data
Hierarchical clustering with the full penguins data
hclust_df <- penguins_clean[,3:6] |>scale() # compute the distancehclust_dist <- hclust_df |>dist()# compute the hierarchical clusteringhclust_results <-hclust(hclust_dist, method ="average")hclust_results
Call:
hclust(d = hclust_dist, method = "average")
Cluster method : average
Distance : euclidean
Number of objects: 333
Hierarchical clustering with the full penguins data
The base R function cutree() can be used to cut the dendrogram into a specified number of clusters.
# cut the dendrogram to get 2 clustershclust_clusters <-cutree(hclust_results, k =2)dt <- penguins_clean |>mutate(cluster =as.factor(hclust_clusters)) dt |>count(cluster, species)
# A tibble: 3 × 3
cluster species n
<fct> <fct> <int>
1 1 Adelie 146
2 1 Chinstrap 68
3 2 Gentoo 119
dt |>ggplot(aes(x = bill_dep, y = flipper_len, color = cluster)) +geom_point()
