z-Score

This tutorial is about z-standardization (z-transformation). We discuss what the z-score is, how z-standardization works and what the standard normal distribution is. It also explains what the z-score table is and what it is used for.

What is z-standardization?

Z-standardization is a statistical procedure used to make data points from different datasets comparable. In this procedure, each data point is converted into a z-score. A z-score indicates how many standard deviations a data point is from the mean of the dataset.

Example of z-standardization

Suppose you are a doctor and want to examine the blood pressure of your patients. For this purpose, you measured the blood pressure of a sample of 40 patients. From the measured data, you can naturally calculate the average, i.e., the value that the 40 patients have on average.

Now one of the patients asks you how high his blood pressure is compared to the others. You tell him that his blood pressure is 10 mmHg above average. Now the question arises: is 10 mmHg a lot or a little?

If the other patients cluster very closely around the mean, then 10 mmHg is a lot in relation to the spread, but if the other patients spread very widely around the mean, then 10 mmHg might not be that much.

The standard deviation tells us how much the data are spread out. If the data are close to the mean, we have a small standard deviation; if they are widely spread, we have a large standard deviation.

Let's say we get a standard deviation of 20 mmHg for our data. This means that a typical distance of the patients' values from the mean is about 20 mmHg.

The z-score now tells us how far a person is from the mean in units of standard deviation. So, a person who is one standard deviation above the mean has a z-score of 1. A person who is two standard deviations above the mean has a z-score of 2. And a person who is three standard deviations above the mean has a z-score of 3.

A person who deviates by minus one standard deviation has a z-score of -1, a person who deviates by minus two standard deviations has a z-score of -2, and a person who deviates by minus three standard deviations has a z-score of -3.

If a person has exactly the value of the mean, then they deviate by zero standard deviations from the mean and receive a score of zero.

Thus, the z-score indicates how many standard deviations a measurement is from the mean. As mentioned, the standard deviation is just a measure of the dispersion of the patients' blood pressure around the mean.

In short, the z-score helps us understand how unusual or typical a particular measurement is compared with the overall distribution.

Calculating the z-score

How do we calculate the z-score? We want to convert the original data, in our case the blood pressure, into z-scores, i.e., perform a z-standardization.

Here we see the formula for z-standardization. In the formula, z is the z-value we want to calculate, x is the observed value (the blood pressure of the person in question), μ is the population mean, and σ is the population standard deviation.

Caution: In our example, we only have a sample. We therefore use the sample mean, usually written as x-bar, and the sample standard deviation, written as s, to standardize the observed sample values.

Let's assume that the 40 patients in our example have a mean value of 130 and a standard deviation of 20. If we use both values, we get for z: x - 130 divided by 20.

Now we can use the blood pressure of each individual patient for x and calculate the z value. Let's just do this for the first patient. Let's say this patient has a blood pressure of 97, then we simply enter 97 for x and get a z-value of -1.65.

This person's value is therefore 1.65 standard deviations below the mean. We can now do this for all patients.

Regardless of the unit of the initial data, we now have an overview in which we can see how far a person deviates from the mean in units of the standard deviation.

Our 40 patients are only a sample from a larger population. If blood pressure in that population is normally distributed and its mean and standard deviation are known, we can use a z-value to determine what percentage of the population has a blood pressure below or above a given value. If we estimate the mean and standard deviation from the sample instead, the resulting percentages are estimates and are subject to sampling uncertainty.

But how does this work? If the initial data are normally distributed, we obtain a so-called standard normal distribution through z-standardization.

The standard normal distribution is a specific type of normal distribution with a mean value of 0 and a standard deviation of 1.

The special feature is that any normal distribution, regardless of its mean or standard deviation, can be converted into a standard normal distribution.

Since we now have a standardized distribution, all we really need is a table that tells us what percentage of the values are below a given z-value.

You can find such a table in almost every statistics book or here: Table of the z-distribution. Now, of course, the question is how to read this table.

If, for example, we have a z-value of -2, then we can read a value of 0.0228 from this table.

This means that 2.28% of the values are smaller than a z-value of -2. As the total is always 100% or 1, 97.72% of the values are greater.

With a z-value of zero, we are exactly in the middle and get a value of 0.5. Therefore 50% of the values are smaller than a z-value of 0 and 50% of the values are greater than 0. As the normal distribution is symmetrical, we can read off the probabilities for positive z-values exactly.

If we have a z-value of 1, we only need to search for -1. However, we must note that in this case the value for -1 tells us what percentage of values are greater than 1. So with a z-value of 1, about 15.87% of the values are larger and about 84.13% are smaller.

But what if, for example, we want to read a z-value of -1.81 in the table? We need the other columns for this. We can read a z-value of -1.81 at -1.8 and at 0.01.

Now let's look at the example about blood pressure again. If we want to know what percentage of patients have a blood pressure below 123, we can use z-standardization to convert a blood pressure of 123 into a z-value; in this case we get a z-value of -0.35.

Now we can take the table of the z-distribution and search for a z-value of -0.35. Here we have a value of 0.3632. This means that 36.32 percent of the values are smaller than a z-value of -0.35 and 63.68 percent are larger.

Compare different data sets with the z-score

There is another important application for z-standardization: it can help to make values measured in different ways comparable. Here is an example.

Suppose we have two classes, class A and class B, who have written a different test in mathematics.

The tests are designed differently, have a different level of difficulty and a different maximum score.

To compare each pupil's performance relative to the scores in their own class, we can apply z-standardization.

The average score or mean score for class A was 70 points with a standard deviation of 10 points. The average score for the test in class B was 140 points with a standard deviation of 20 points.

We now want to compare the performance of Max from class A, who scored 80 points, with the performance of Emma from class B, who scored 160 points.

To do this, we simply calculate the z-value of Max and Emma. We enter 80 once for x and get a z-value of 1. Then we enter 160 for x and also get a z-value of 1.

The z-values of Max and Emma are therefore the same. This means that both students performed equally well relative to the scores in their respective classes. Both are exactly one standard deviation above the mean of their class. This comparison is meaningful only if the two tests measure sufficiently similar skills.

Assumptions

What about the assumptions? Can we simply calculate a z-standardization and use the table of the standard normal distribution?

The z-standardization itself, i.e. the conversion of the data points into z-values using this formula, does not require a normal distribution. It can be carried out for any numerical data with a standard deviation greater than zero.

However, if we interpret the resulting z-values using probabilities from the standard normal distribution, the underlying variable must be normally distributed.

If individual values from a normally distributed population are standardized using the population mean (μ) and population standard deviation (σ), their z-values follow the standard normal distribution.

In practice, the population mean and standard deviation are usually unknown and must be estimated from a sample. Probabilities based on these estimates are therefore approximate and depend on how well the sample represents the population.

The central limit theorem concerns sample means, not the distribution of individual observations. Under broad conditions, the sampling distribution of the mean becomes approximately normal as the sample size increases. It does not mean that a sample or its individual z-scores become normally distributed once the sample size is greater than 30. Whether a normal approximation is adequate depends on the population distribution and the sample size; 30 is only a common rule of thumb, not a universal cutoff.

When the standard deviation is estimated from the sample, s is usually written instead of σ and x-bar instead of μ for the mean.

The z-standardization should not be confused with the z-test or the t-test. If you want to know what the t-test is, please watch the following video.