Repeated Measures ANOVA

What is a repeated measures ANOVA?

Repeated measures ANOVA tests whether there are statistically significant differences in the means of three or more dependent conditions. In a repeated measures design, the same participants are measured multiple times under different conditions or at different time points.

The one-way analysis of variance with repeated measures is the extension of the t-test for dependent samples for more than two groups.

In the t-test for dependent samples, we examined whether there is a difference between two dependent samples. If we want to test whether there is a difference between more than two dependent samples, we use the analysis of variance with repeated measures.

What are dependent samples?

In a dependent sample, the measured values are connected. For example, if a sample is drawn of people who have knee surgery and these people are interviewed before the surgery and one week and two weeks after the surgery, it is a dependent sample. This is the case because the same person was interviewed at three points in time.

Now, of course, it doesn't have to be about people or points in time. More generally, we can say: In a dependent sample, the same test units are measured several times under different conditions. The test units can be people, animals or cells, for example, and the conditions can be time points or treatments, for example.

Difference of analysis of variance with and without repeated measurements

If three or more independent samples are available, ANOVA without repeated measures is used. But be careful, of course the assumptions have to be checked! More about this later!

Example of repeated measures ANOVA

You might be interested to know whether a patient's perception of pain changes during therapy after a slipped disc. For this purpose, you measure pain perception before the therapy, in the middle of the therapy and at the end of the therapy. Now you want to know if there is a difference between the time points.

So, your independent variable is time, or therapy progressing over time. Your dependent variable is the pain perception. You now have a history of the pain perception of each person over time and want to know whether mean pain perception changes during therapy. Without a comparison group or another suitable design, a change over time cannot by itself prove that the therapy caused the change.

To put it simply, the left plot shows pain changing over time, whereas the right plot shows no change over time. The ANOVA tests whether the mean differences between time points are larger than would be expected from random variation.

Research question and hypotheses

What is the research question in a repeated measures ANOVA? The research question is: Is there a difference between the population means at the different time points or conditions?

The null and alternative hypotheses result in:

• Null hypothesis: the population means are equal at all time points or conditions.
• Alternative hypothesis: at least one population mean differs from another.

Assumptions ANOVA with repeated measures

Now we come to the assumptions of ANOVA with repeated measures and finally I will show you how you can easily calculate it online. So what are the assumptions?

• Dependent samples: The samples must be dependent samples.
• Metric scale level: The dependent variable must be metric.
• Independent participants: Although measurements from the same participant are related, different participants should be independent of one another.
• Normality: The model residuals should be approximately normally distributed. This assumption is especially important with small samples; larger samples generally make the ANOVA more robust to moderate deviations from normality.
• Sphericity: The variances of the differences between all combinations of the different groups should be equal. This assumption can be tested using Mauchly's test of sphericity.
• No significant Outliers: Outliers can have a disproportionate effect on ANOVA, potentially leading to misleading results.

For example, a QQ plot or a normality test can help assess normality. A mild departure does not automatically make repeated measures ANOVA invalid. For strongly non-normal data or severe outliers, the Friedman test may be an appropriate alternative.

Whether the assumption of sphericity is violated can be tested using Mauchly's test for sphericity. A p-value below the chosen significance level provides evidence that sphericity is violated. A larger p-value means that the test did not detect a violation, but it does not prove that sphericity holds, especially in a small sample.

If this assumption is violated, adjustments such as Greenhouse-Geisser or Huynh-Feldt can be applied to the degrees of freedom and p-value. Sphericity is automatically satisfied when there are only two time points or conditions.

Results of the repeated measures ANOVA

The analysis of variance with repeated measurement calculates a p-value for your data. This p-value measures how compatible the observed differences are with the null hypothesis that all population means are equal.

If the calculated p-value is smaller than the predefined significance level, which is usually 0.05, the null hypothesis is rejected. In this example, the p-value is 0.01, which is less than 0.05. Therefore the null hypothesis is rejected, providing evidence that at least one time-point mean differs. The omnibus ANOVA does not show which time points differ.

Effect size for repeated measures ANOVA

In a repeated measures ANOVA, the effect size can be calculated via the partial eta squared (η_p²). It compares the sum of squares for the repeated-measures effect with the sum of squares for that effect plus its error. Larger values indicate a larger effect, but the study context and confidence interval should also be considered.

Bonferroni Post-hoc-Test

If the omnibus test is significant, it is often useful to identify which time points differ. This can be done with Bonferroni-adjusted pairwise comparisons.

These comparisons use paired t-tests because the measurements are dependent. Testing many pairs increases the probability of at least one Type I error across the family of tests. The Bonferroni method controls this risk by multiplying each unadjusted p-value by the number of comparisons, with adjusted p-values capped at 1.

In the present case, 3 tests were performed, so for the calculation of the Bonferroni post-hoc test, the p-value obtained from the t-test was multiplied by 3 in the background. If one or more p-values are less than 0.05, a significant difference between the two groups is assumed. In this case, we therefore have a significant difference between Before and End and between Middle and End.

Calculate repeated measures ANOVA with numiqo

ANOVA with repeated measures can be easily calculated with numiqo. To do this, simply visit the repeated measures ANOVA calculator numiqo and copy your own data into the table.

Now you just need to select your variables. If you select three or more metric variables, an analysis of variance with repeated measures is automatically calculated.

And you get the results. You can read the p-value in the table and if you don't know exactly how to interpret the results, just click on interpretation in words. In addition, the results are displayed in a boxplot. Finally, the Bonferroni post-hoc test is calculated.

Calculate a repeated measures ANOVA by hand

How do you calculate an analysis of variance with repeated measures by hand? Here you can find the formulas to calculate an ANOVA. Let's say this is our data. We have 8 people, each of whom we measured at three different points in time (start, middle and end).

First we can calculate the necessary mean values. With the mean values we can calculate the sums of squares and the mean squares. Now we can calculate the F-value, which is calculated by dividing the mean square of the treatment by the mean square of the residual or error. Finally, we can calculate the p-value using the F-value, the degrees of freedom for the treatment and error, and the F-distribution.