Friedman Test

What is the Friedman test?

The Friedman test is a non-parametric statistical test used for analyzing repeated measures or matched data. It is often used when the normality assumptions of a repeated measures ANOVA are not reasonable. It can also be used with ordinal data, making it an alternative to repeated measures ANOVA.

What is a dependent sample (repeated measure)? In a dependent sample, the measured values are connected. For example, if a sample consists of patients who have undergone knee surgery and these people are each surveyed before the surgery and one and two weeks after the surgery, it is a dependent sample. This is the case because the same person was interviewed at multiple time points.

Friedman Test vs. ANOVA with repeated measures

You might rightly say that the analysis of variance with repeated measures tests exactly the same thing, since it also tests whether there is a difference between three or more dependent samples.

Yes, the Friedman test is the non-parametric counterpart of the analysis of variance with repeated measures. But what is the difference between the two tests?

Repeated measures ANOVA compares means using the measured values. The Friedman test, meanwhile, uses ranks within each person or matched set rather than the actual measured values.

In this example, the time point at which a person has the highest value gets rank 1, the time point with the second-highest value gets rank 2, and the time point with the smallest value gets rank 3. This is done for every person or matched set. If values are tied within a row, they receive the average of the ranks they would occupy. Afterwards, the ranks for each time point are added up.

At the first measurement occasion, we get a sum of 7, at the second time we get a sum of 8 and at the third time we get a sum of 9. Now we can check how much these rank sums differ.

Why are ranks used? An advantage of comparing ranks rather than means is that the measurements do not have to be normally distributed.

In practice, the choice also depends on the scale of measurement and the study design. Repeated measures ANOVA is commonly used for continuous data when its assumptions are reasonable. The Friedman test is suitable for ordinal data and is a common alternative for non-normal continuous data with three or more repeated or matched measurements.

Hypotheses in the Friedman test

This brings us to the research question, which you can answer with the Friedman test. The research question is whether the distributions differ among three or more dependent groups. The null and alternative hypotheses are therefore:

• Null hypothesis: the distributions are the same across the dependent groups.
• Alternative hypothesis: at least one group has a different distribution.

As noted previously, the Friedman test does not use the true values, but the ranks.

If the distributions have a similar shape, this result is often interpreted as a test of whether their locations, such as their medians, differ. Measurements within a person or matched set are related, but the people or matched sets should be independent of one another.

Friedman test example

You might be interested to know whether therapy after a herniated disc has an influence on the patient's perception of pain. For this purpose, you measure the pain sensation before the therapy, in the middle of the therapy and at the end of the therapy. Now you want to know if there is a difference between the different time points.

So, your independent variable is time, or the progress of the therapy over time. Your dependent variable is the perception of pain. You now have a progression of pain perception from each person over time and now you want to know if the therapy is associated with changes in pain perception.

Put simply, one person's pain score changes over time while another person's score does not. The test examines whether there is an overall pattern of change across the people studied; by itself, it does not prove that the therapy caused the change.

Calculate Friedman test

Let's say you want to investigate whether there is a difference in the responsiveness of people in the morning, at noon and in the evening. For this purpose, you measured the reactivity of 7 people in the morning, at noon and in the evening.

In the first step we have to assign ranks to the values. For this we look at each row separately.

In the first row, or in the first person, 45 is the largest value, this gets rank 1, then comes 36 with rank 2 and 34 with rank 3. We now do the same for the second row. Here 36 is the largest value and gets rank 1, then comes 33 with rank 2 and 31 with rank 3. We now do this for each row.

Afterwards we can calculate the rank sum for each time of the day, so we simply sum up all ranks at each column. In the morning we get 17, at noon 11 and in the evening 14.

Under the null hypothesis, the expected rank sum is the same at every time point. The expected value is obtained with the first equation in the image and, in this case, is 14. Thus, if there is no systematic difference between morning, noon, and evening, we would expect a rank sum of 14 at all three time points.

Next, we calculate the Chi² value using the second equation in the image. N is the number of people, i.e. 7, k is the number of time points, i.e. 3, and the sum of R² is 17² + 11² + 14². Thus, we get a Chi² value of 2.57. When tied values occur, the statistic requires a tie correction.

Now we need the number of degrees of freedom. This is given by the number of time points minus 1, so in our case 2.

At this point we can read the critical Chi² value in the critical values table. For this we take the predefined significance level, let's say it is 0.05 and the number of degrees of freedom. We can read that the critical Chi² value is 5.99. This is greater than our calculated value. Thus, the null hypothesis is not rejected; these data do not provide sufficient evidence of a difference in responsiveness across the time points. If the calculated Chi² value were greater than the critical value, we would reject the null hypothesis. This Chi² method is an approximation; for small samples, an exact or permutation p-value may be preferable when available.

Calculate Friedman test with numiqo

For the calculation of the Friedman test you can simply use numiqo. To do this, simply go to the Friedman test calculator on numiqo.com and copy your own data into the table.

Now we get the results for the Friedman test.

First you get the descriptive statistics. Then you can read the p-value. If you don't know exactly how to interpret the p-value, you can simply look at the interpretation in words. The result can be reported as: "A Friedman test found no statistically significant difference across the measurement occasions, Chi² = 2.57, p = 0.276."

If your p-value is greater than your chosen significance level, the null hypothesis is not rejected. Usually, a significance level of 0.05 is used, so this p-value is greater. This means that the data do not provide sufficient evidence of a difference; it does not prove that the distributions are identical.

Post-Hoc Test

In addition, numiqo provides a post-hoc test. If the overall Friedman test is statistically significant, post-hoc pairwise comparisons can examine which specific groups differ from one another.

Here, two dependent groups are compared in each row. The Bonferroni-adjusted p-value is obtained by multiplying the unadjusted p-value by the number of pairwise comparisons, with values above 1 reported as 1. This adjustment controls for the increased chance of a false-positive result when several comparisons are performed.

If an adjusted p-value is less than the chosen significance level, the corresponding pair provides evidence of a difference.