Hierarchical Cluster Analysis

A hierarchical cluster analysis is a clustering method that creates a hierarchical tree or dendrogram of the observations to be clustered. Agglomerative clustering starts with separate observations and merges them, whereas divisive clustering starts with one cluster and splits it. This tutorial uses the agglomerative approach.

The tree represents the relationships between objects and shows how objects are clustered at different levels. A horizontal cut through the dendrogram defines a possible cluster solution: the branches below the cut form the clusters.

Example Hierarchical Cluster Analysis

In our example we asked people about how many hours per week they spend on social media platforms and at the gym.

We now want to know if there are clusters in this dataset and perform a hierarchical cluster analysis.

How is a Hierarchical Cluster Analysis calculated?

First, we plot the points in a scatter plot.

We can now start to create the clusters. In the first step, each point forms its own cluster. We therefore have as many clusters as people.

The goal is to merge clusters step by step until all points are in one cluster.

In each step, the clusters that are closest together are merged. What does "closest together" mean?

To do this, we need to determine two things:

• How the distance between two points is measured.
• How the distance between two clusters is defined by a linkage method.

Distance between two points

Let's start with the question of how to calculate the distance between two points. Here are some common distance measures for numerical variables:

• Euclidean Distance
• Manhattan Distance
• Maximum Distance

Variables measured on larger scales can dominate these distances. Standardizing variables is therefore often useful when their units or ranges differ. These distances are not generally suitable for categorical variables; mixed data require an appropriate distance measure.

Let's take the distance between Max and Caro. The difference on the y-axis is 1 and the difference on the x-axis is 4.

Euclidean Distance

The Euclidean distance is the square root of the sum of the squared differences.

Manhattan Distance

The Manhattan distance uses the sum of the absolute differences. So we simply calculate 4 plus 1, which gives a distance of 5.

Maximum Distance

The maximum distance is simply the maximum value of the absolute differences. In this case it is 4.

Linkage Methods

Now that we know how to calculate distances between points, we need to determine how to link the points within a cluster.

Let's say we have a cluster with the points Joe and Lisa and a cluster with Max and Caro. Now how do we determine the distance between these two clusters? Here are the most popular methods:

• Single linkage
• Complete linkage
• Average linkage

Single-linkage

Single linkage uses the distance between the closest elements in the clusters. This is the distance between Caro and Joe.

Complete-linkage

Complete linkage uses the distance between the farthest elements in the clusters. So between Max and Joe.

Average-linkage

Average linkage uses the average of all pairwise distances. The distance is calculated for each pair and then averaged.

Example: Hierarchical Cluster Analysis

For our example, we use the Euclidean distance and the single linkage method. So now we need the distance from each cluster to all other clusters.

For this we first need to calculate the distance matrix. In the distance matrix we enter the clusters on both dimensions and then calculate the distances from each cluster to every other cluster.

The distance between Alan and Lisa is given by:

We can now do this for all other combinations until we have calculated the full distance matrix. Now we can merge the first clusters by finding the smallest distance. This is the case between Joe and Lisa.

With this, we now combine Joe and Lisa into one cluster. In our tree diagram or dendrogram we can draw the first connection.

Now we need to update our distance matrix. We decided to use the single linkage method. Therefore, the distance between two clusters is the smallest distance between any point in one cluster and any point in the other. In this example, Joe is the closest member of the Lisa-and-Joe cluster to Alan, Max, and Caro.

So we calculate the distance from Alan to Joe, the distance from Max to Joe, and the distance from Caro to Joe.

Now we again merge the clusters that are closest. These are Max and Alan.

In our tree diagram or dendrogram, we can draw the second connection.

Now we update the distance matrix again. For every pair of clusters, we retain the smallest distance between their members. The smallest updated distance is between the Caro cluster and the Lisa-and-Joe cluster.

So we connect these two clusters and draw the third connection in the tree diagram.

Now there are only two clusters left, and we merge them in the last step to obtain the finished dendrogram.

Calculate hierarchical cluster analysis with numiqo

To calculate a hierarchical cluster analysis online, just visit the statistics calculator and copy your own data into the table or use the link to load the dataset. Now we click on Cluster and select Hierarchical Cluster.

If we now click on Social Media and Gym, a hierarchical cluster analysis will be calculated for us. Additionally, we can specify the label, in our case the names of the people.

Now we can specify which linkage method should be used and how the distance should be calculated. We simply take single linkage and the Euclidean distance again.

Now we get the results below: the tree plot, a scatter plot, and the elbow plot. In the elbow plot we can look for a bend that suggests a possible number of clusters. Here, the bend suggests four clusters. After selecting four, the tree plot highlights them in different colors.

The number of clusters is not determined by a formal significance test. It should also be checked against the dendrogram, subject knowledge, and whether the clusters remain reasonably stable when the data or analysis settings change. Outliers can form their own clusters or change later merges, and different distance or linkage choices can produce different solutions. Clusters describe patterns in this dataset; they are not proof of naturally distinct populations or causal relationships.