Degrees of Freedom (df) in Statistics

This tutorial is about the calculation, use and interpretation of degress of feedom in statistics.

What are Degrees of Freedom in general?

Degrees of freedom are the number of independent pieces of information you have left to vary when estimating something.

But in order to understand degrees of freedom, we’ll first take a quick tour to a place where degrees of freedom are very easy to understand, and then we’ll apply this to statistics. So what’s the trick to making degrees of freedom make sense in statistics? We’ll start with mechanics.

Degrees of Freedom Example in Mechanics

In Mechanics, the degrees of freedom tell us how many independent ways something can move.

If we have a point, and this point can move left and right, we have 1 degree of freedom. If this point can move up and down we have a second degree of freedom. And if this point can move back and forth; we have a third degree of freedom.

For example a helicopter moving through space has three degrees of freedom for translation: left–right, up–down, and forward–back. Yes, it can also rotate, but we’ll ignore rotation here.

But what if we look at an elevator? An elevator just can move up and down, because it’s constrained by the shaft. Therefore, because it can just move up and down, it has only one degree of freedom.

If we have two elevators, each one can move up or down. If we treat them as one system, we have two degrees of freedom in total.

If we have three elevators we there are three degrees of freedom, each can move up or down.

So, let’s take it a step further with another example, which then will bring us close to what degrees of freedoms are in statistics.

Degrees of Freedom Example

Imagine in a city you've got a small hill with two funiculars moving up and down. The cars are linked by a cable. When one goes up, the other must go down.

So normally, two independent cars would give the system two degrees of freedom—like two elevators. But the connecting cable is a constraint, so the system loses one degree of freedom.

Imagine three cars connected by a single cable. Because the cable length is fixed, if the first car moves up by some distance, the combined downward movement of the second and third must equal that same distance.

For example, the second car could move down exactly that amount while the third stays put—or they could split it between them in any proportion.

But of course, we can also write this constraint mathematically. If we measure each car’s height from the ground, the sum of their heights is constant. For example, the heights h1 plus h2 plus h3 might always add up to 300 m.

Because one equation ties ℎ1,ℎ2,and ℎ3 together, we lose one degree of freedom.

With three variables and one constraint, we have 3−1= degrees of freedom, so we have 2 degress of freedom.

So, in mechanics you can literally see the constraint—the cable—and you can also write it as an equation.

In statistics, on the other hand, it is difficult to imagine the constraints. But now that we’ve built the picture in mechanics, let’s carry it over to statistics.

What are Constraints in Statistics?

In Statsitics, or more preceicly in hypothesis testing, we are using a sample, to make a statement about the whole population. But here is the key point: we use this sample twice.

First, we use the sample to test our claim. Second, we use that same sample to estimate things—like the mean. Hmm what does that mean?

Let’s walk through a t-test and see how degrees of freedom work.

What are the Degrees of Freedom in a t-test?

The t-test—more precisely, the independent-samples t-test —checks whether two groups, for example group A and group B differ significantly.

So first we need two samples. If we collect 8 people in each group, we have 8 independent data points in each group.

But to test whether these two samples differ, we use each group’s mean. So we compute a mean for each group and, to calculate it, we use the same data we’re testing.

And as we’ll see shortly, these means are like the cable in our mechanical example—they tie the data points together and give us a constraint.

If, for example, we have a sample of seven people and we calculate the mean from this data, only 6 of them can vary freely. If we know the mean and the first 6 values, we automatically know the value of the 7th person.

Like in the mechanics example: if the cars are connected with a cable and we know the position of the first 2 cars, we automatically know the position of the last one.

So in this case we have two groups and we calculate the mean of both samples. We have two independent constraints and because of these two constraints, we lose two degrees of freedom.

Therefore, in an independent-samples t-test, the degrees of freedom are 𝑛_1+𝑛_2−2, where 𝑛_1and 𝑛_2 are the sample sizes of the two groups and the 2 results because of the two means.

Or, if we have 9 data points and want to fit a simple linear regression, we need to estimate two parameters—the slope and the intercept. That costs 2 degrees of freedom. Therefore, for a simple linear regression, the degrees of freedom are the number of data points minus 2.

So in statstics, degrees of freedom tell you how many pieces of information in your data are still “free to vary” after you’ve used some of them to estimate things (like a mean).

Why do we need Degrees of freedom ?

In mechanics, degrees of freedom are intuitive: they usually follow from the system’s requirements. For cable cars, when one car goes down it should pull the other up, so we connect them with a cable. This constraint reduces how each car can move.

In statistics, we talk about degrees of freedom because our methods introduce constraints. We use the sample to test a claim and also to estimate quantities (like means or variances). Each estimate “uses up” a degree of freedom and limits how the data can vary independently.

In hypothesis testing, we compute a p-value using a reference distribution—such as the t distribution, chi-square distribution, or F distribution.

And these distributions depend on the degrees of freedom. For example, this is the t-distribution for different degrees of freedom

If you use the wrong degrees of freedom, you get the wrong curve, the wrong p-value, and possibly the wrong conclusion.