Wilcoxon signed-rank test

What is a Wilcoxon signed-rank test?

The Wilcoxon signed-rank test is a non-parametric statistical test used to determine whether two dependent (paired) samples differ significantly from each other. Instead of comparing means (t-test), the Wilcoxon test is based on the ranks of the paired differences.

The Wilcoxon signed-rank test is a non-parametric alternative to the paired samples t-test and does not require the paired differences to be normally distributed. However, it has its own assumptions, which are described below.

Assumptions of the Wilcoxon test

The Wilcoxon signed-rank test is a non-parametric alternative to the paired t-test, especially when the normality assumption is not reasonable. The following assumptions should be considered:

If the data are not available in pairs, the Mann-Whitney U test is used instead of the Wilcoxon test.

Hypotheses in the Wilcoxon test

The hypotheses are stated for the paired differences. When their distribution is continuous and symmetric, the test can be interpreted as testing whether the median paired difference is zero:

One-sided alternatives can be used when a direction was specified before examining the data. Without the symmetry assumption, the signed-rank test is not simply a test of medians.

Wilcoxon test and test power

One might wonder: why not always use the Wilcoxon test instead of the t-test for dependent samples? That way, there's no need to check for normality!

When the paired differences are approximately normal, the paired t-test is usually more powerful for detecting a difference in means. For skewed or heavy-tailed data, however, the Wilcoxon test can perform as well as or better than the t-test. The choice should therefore depend on the research question, measurement scale, and distribution of the paired differences.

Calculation of the Wilcoxon test

To perform the Wilcoxon test for two dependent samples, first, calculate the differences between the paired values. Then, take the absolute values of these differences and rank them accordingly. It is crucial to retain the original signs of the differences throughout the process. Pairs with a difference of zero are usually excluded, and tied absolute differences receive their average rank. (An example with tied ranks follows.)

In the final step, the rank sums are computed separately for the positive and negative differences.

Here, the test statistic W is the sum of the positive ranks. Notation varies between textbooks and software: some report the positive rank sum, while others report the smaller of the positive and negative rank sums. Always check which definition is being used.

In this example, the test statistic W is 8. If there is no difference between the two dependent samples, the expected value can be calculated using the following formula:

Next, the test statistic W is compared with its expected value by calculating the standardized test statistic z. For this, we need its standard deviation.

Now we have everything needed to calculate the z value.

We can now calculate the p-value corresponding to the z statistic and compare it with the chosen significance level.

Note: For small samples, an exact p-value may be available. For larger samples, software commonly uses a normal approximation, with adjustments for tied ranks and zero differences when needed. The sample-size cutoff varies between methods and software.

For a two-sided hypothesis, the one-tailed p-value (0.3) is multiplied by two, resulting in a final p-value of 0.6.

Continuity Correction

Load Example Data

Many statistical software programs, such as numiqo, apply a so-called continuity correction in the normal approximation for the p-value. As a result, the p-value may vary slightly.

And here is the entire calculation workflow presented in a single figure:

Calculate Wilcoxon signed-rank test with tied ranks

If several observations share a rank, tied ranks are present. In this case, there is a change in the calculation of the rank sums and the standard deviation of the W-value. We will now go through both using an example.

In the example it can be seen that there are...

• ...three observations with an absolute difference of 2; these observations share ranks 2, 3 and 4.
• ...two observations with an absolute difference of 4; these observations share ranks 6 and 7.

To account for these tied ranks, the mean of the tied ranks is calculated in each case. In the first case, this results in a "new" rank of 3 and in the second case in a "new" rank of 6.5. Now we can calculate the rank sums of the positive and negative ranks.

Since the rank ties are clearly visible in the table above, a correction term is calculated for the standard deviation of W. All values are now available to calculate the z-value while accounting for tied ranks.

For small samples, use an exact method when it is available. For larger samples, the normal approximation can be used; the appropriate cutoff depends on ties, zero differences, and the software's method.

Effect size in the Wilcoxon signed-rank test

An effect size describes the magnitude of the observed difference. There are several measures to calculate the effect size in the Wilcoxon test. A common method is to use r, defined as:

Here, z is the standardized test statistic from the Wilcoxon test and n is the number of pairs included in the test, usually after excluding zero differences.

The value of r can range from -1 to 1. Values near 0 indicate a small effect, while values near -1 or 1 indicate a strong effect. The sign indicates the direction of the effect and depends on the order in which the paired difference was calculated.

The following table can be used to interpret the effect size (effect size r according to Cohen (1988)).

Calculation of the Wilcoxon test with numiqo

A Wilcoxon test can easily be calculated with numiqo. Simply copy the table below or your own data into the Statistical Calculator and click on Hypothesis tests. Then click on the two variables and select Non-Parametric Test.

numiqo then gives you the following result.

If you want to analyze more than two dependent variables, the Friedman test should be used instead. You can easily calculate a Friedman test online. To do this, simply click on more than two metric variables.