Paired Samples t-test

What is the Paired Samples t-test?

A paired-samples t-test (also called a dependent t-test) examines whether the mean difference between paired measurements is zero. The pairs may be repeated measurements from the same people or objects, or measurements from matched individuals.

When do you use it?

Whenever the values belong together in pairs, e.g.:

• Before and after: Measure the weight of a group before a diet and then again after the diet.
• Two conditions: Test how quickly the same group of people responds to a stimulus with coffee and then without coffee.
• Related pairs: Compare the opinions of husbands and wives (who are linked as a pair).

The central question of this test is: Is the mean difference we observe in our sample strong enough to suggest a difference in the "real world" (population), or is it simply the result of random fluctuation?

In contrast to the independent-samples t-test, where you compare two separate groups (like men vs. women), the paired t-test focuses on differences within pairs.

Why do you need the dependent t-test?

You use the paired-samples t-test when observations belong together in pairs, for example, when the same people are measured before and after an intervention.

For example, you might want to know whether a new medication lowers blood pressure. Since you can't measure everyone who takes the medication, you take a random sample. You can then use a paired t-test to assess whether mean blood pressure changed after the medication in the broader population. Whether the change can be attributed to the medication also depends on the study design.

What are dependent or paired samples?

In dependent samples, these measured values are available in pairs. The pairs result from repeated measurements or matching. This can be the case, for example, in longitudinal studies with several measurement points (time series analyses) or in intervention studies with experimental designs (before-after measurement).

An example of dependent sampling is when the weight of a group of people is measured at two points in time. A person can then be uniquely assigned a weight at the first and second measurement time points and the difference in the measured values can be calculated in each case. If more than two measurement times are available, a method such as repeated-measures ANOVA may be used instead.

What is the advantage of a dependent t-test over an independent t-test?

The question of whether to use a dependent t-test or an independent t-test is, of course, already determined as part of the study design, and it is not possible to arbitrarily use either one test or the other. Therefore, the question is rather which type of study makes more sense:

• Conducting a study with one group of participants who are measured twice.
• To conduct a study with two separate groups of participants, each measured once.

The major advantage of a repeated-measures design that then uses the paired t-test is that stable differences between participants can be controlled by comparing each person with themselves. When the paired measurements are positively correlated, this often gives the paired t-test greater statistical power than an independent t-test.

Example of a paired sample t-test

The t-test for dependent samples has numerous applications, here are three examples.

Research Question and hypotheses

In order to be able to calculate a t-test for dependent samples, you first need to define a question and the hypotheses.

Research Question

In a t-test for dependent samples, the general question is: Is there a statistically significant mean difference between the paired measurements?

The questions for the above examples arise as follows:

• Does the new drug help to increase memory performance?
• Does the newly introduced lubricant have an influence on downtimes?
• Has the health consciousness of the German population changed between 2010 and 2015?

Hypotheses

Now the hypothesis can be derived from the question. In the hypothesis, a preliminary, i.e. unsubstantiated, assumption is made which is to be tested. In the case of a t test for dependent samples, the hypotheses are:

• Null hypothesis H₀: The population mean of the paired differences is zero.
• Alternative hypothesis H₁: The population mean of the paired differences is not zero.

Assumptions paired t-Test

The assumptions should be checked before calculating the paired t-test. If the outcome is ordinal or the difference scores are strongly non-normal, the Wilcoxon signed-rank test may be suitable. It is a non-parametric counterpart of the paired t-test.

How does a paired t-test work?

In the paired t-test, the difference is calculated from each paired case. The mean value is then calculated from these differences. The test compares the mean difference with its standard error. A larger absolute t-value provides stronger evidence against a population mean difference of zero.

Note

The t-test for dependent samples is very similar to the t-test for one sample. We can also think of the t-test for dependent samples as having a sample that was measured at two different times. As shown in the following image, we then calculate the difference between the paired values and get a value for one sample.

Once we get -5, once +2, once -1 and so on. Now we want to check whether the mean of the just calculated differences deviates from a reference value. In this case, zero. And that is exactly what the t-test does for a sample.

Calculate t-test for paired samples

For the calculation of the t-test for dependent samples, the difference of each pair from the two groups is first formed. From the resulting differences, the mean value x̄_diff is then calculated.

The calculation of the test statistic t is the same as for a one-sample t-test. If there is no difference between the two groups, the mean value of the difference x̄_diff is zero. So the question is, is there a difference between x̄_diff and zero. The test statistic t for the t-test for dependent samples is then calculated as:

where is the standard error of the mean value

• = Difference between the groups
• = Mean value of the difference between the two groups
• = Sample size
• = Standard deviation
• = Estimated standard error of mean

The paired t-test has n - 1 degrees of freedom, where n is the number of pairs.

Effect size dependent t-test

An effect size helps describe the magnitude of the mean change. The following formula calculates Cohen's d for paired samples by dividing the mean difference by the standard deviation of the differences:

Common rough guidelines for interpreting the absolute value of this version of Cohen's d are:

• Effect size d: 0.2 small effect
• Effect size d: 0.5 medium effect
• Effect size d: 0.8 large effect

Calculate with numiqo

In the paired t-test example it is examined whether the summer holidays have an effect on the physical fitness of statistics students.

So the question is: Do the summer holidays have an effect on the physical fitness of statistics students? For this purpose, a fitness test is carried out once before and once after the holidays for 10 statistics students (2 measurement points).

Since two test results always come from one student, there is a dependency between the two samples. Therefore the paired t-test is used.

After copying the upper table into the dependent t-test calculator you can calculate the t-test. The results are as follows:

Statistics

Correlation

Paired t-test

95% confidence interval of the difference

Interpret t-test for dependent samples

If the calculated p-value is smaller than the specified significance level (usually 5%), the null hypothesis is rejected; otherwise, it is not rejected. For the example above, you can report the results as follows:

The score of the variable before the vacations had lower values (M = 46.4, SD = 11.452) than the score of the variable after the vacations (M = 49.7, SD = 14.095). A t-test for dependent samples showed that this difference was not statistically significant, t(9) = -1.392, p = .197, 95% confidence interval [-8.664, 2.064].

The p-value of 0.197 is above the defined significance level of 0.05. The result is therefore not statistically significant, and the null hypothesis is not rejected. The 95% confidence interval includes zero, so the data remain compatible with both a decrease and an increase in the population mean score.