Menu
numiqo
Statistics made easy

Statistics made easy

8th revised edition (March 2026) - many illustrative examples - only €8.99

Free sample

p-value

Author: Dr. Hannah Volk-Jesussek
Updated:

What is the p-value?

The p-value is the probability, assuming the null hypothesis and the statistical model are correct, of obtaining a test statistic at least as extreme as the one observed.

The p-value is used to decide whether the null hypothesis is rejected or not rejected. If the p-value is smaller than the pre-specified significance level (often 5%), the null hypothesis is rejected; otherwise, it is not rejected.

You want to test a claim about the population and have set up a hypothesis for this. Since it is usually not possible to survey the entire population, you survey a sample. This sample will most likely deviate from the population due to chance.

p-value population sample

If the null hypothesis applies in your population, for example that the salary of men and women does not differ, then there will typically be some difference in the sample, such as a difference of EUR 300 or more per month. The p-value tells you how likely a result at least this extreme would be if there were no difference in the population. What counts as equally or more extreme is determined by the chosen test statistic, which also accounts for factors such as sample size and variability.

If the result is a very small probability, you can ask yourself whether the assumption about the population is true at all.

If p = 0.03, then assuming no true difference and that the test assumptions hold, a test result at least as extreme as the observed one would occur in about 3% of repeated samples.

Example

The p-value is used to either reject or not reject the null hypothesis in a hypothesis test. If the calculated p-value is smaller than the significance level, which in most cases is 5%, then the null hypothesis is rejected; otherwise it is not rejected.

p-value

Example:

  • The null hypothesis is that there is no difference between the salaries of men and women.
  • Now a sample is taken with the salary of men and women. These are our observed results.
  • We assume that the null hypothesis is true, that is, that there is no difference between the salaries of men and women.
  • In the observed sample we find that men earn 150 EUR more per month than women.
  • The p-value now indicates how likely it is to obtain a test result at least as extreme as the observed one if there is no difference in the population.
  • If the p-value is, for example, 0.04, a test result at least as extreme as the observed one would occur in 4% of repeated samples if there is no difference in salary in the population and the test assumptions hold.

A p-value of 0.04 does not mean that there is a 4% probability that the null hypothesis is true. It describes the probability of the data, or more extreme data, under the null hypothesis.

The probability of 4% is very low, so one can ask whether it is true that men and women in the population earn the same or whether this hypothesis should be rejected.

The question of when the null hypothesis is discarded is answered by the significance level.

Significance Level

The significance level is determined before the test. If the calculated p-value is below this value, the null hypothesis is rejected; otherwise, it is not rejected. As a rule, a significance level of 5% is chosen.

  • p ≤ 0.01: statistically significant at the 1% level
  • 0.01 < p ≤ 0.05: statistically significant at the 5% level
  • p > 0.05: not statistically significant at the 5% level

The significance level α controls the long-run Type I error rate: when the null hypothesis is true and all test assumptions hold, a procedure with α = 0.05 will reject it in about 5% of repeated studies. It does not mean that a rejected null hypothesis has a 5% probability of being true. Reducing α to 0.01 lowers the Type I error rate, but also makes it harder to detect a real effect.

One-tailed p-value

Let's say you are examining the reaction time of two groups. Often it is not only whether there is a difference between the two groups that is of interest, but whether one group has a larger or smaller value than the other. In this case, you have a directional hypothesis and calculate a one-sided p-value.

A one-tailed p value includes values more extreme than the obtained result in one direction, that direction having been stated in advance.

A two-tailed p value includes values more extreme in both positive and negative directions.

For many common tests with symmetric test-statistic distributions, the one-sided p-value is half the two-sided p-value when the observed effect is in the direction specified in advance. This shortcut does not apply to every test, so the one-sided p-value should normally be calculated directly by the statistical software.

Example

Your alternative hypothesis is that group A has greater reaction time values than group B. When analyzing your data you get a two-sided p-value of 0.04.

Now you have to check whether group A really has larger values in your data. If this is the case, the two-sided p-value is divided by two, so you get 0.02.

If this is not the case and the effect or difference goes in the opposite direction from the alternative hypothesis, your p-value is 1 - 0.02, i.e. 0.98.

Don't worry, if you use numiqo, you can specify what kind of hypothesis you have, and numiqo will help you evaluate it.

Calculate p-value

To calculate the p-value a suitable hypothesis test must first be found. After selecting the appropriate hypothesis test, you can calculate the p-value in the numiqo statistics calculator. The best known hypothesis tests are:

To calculate the p-value, the null distribution of the test statistic is needed. This distribution shows how the test statistic would vary in repeated samples if the null hypothesis were true. Examples include the t-distribution and the Chi-square distribution.

Chi-square distribution

p-value statistics

Statistical tests and the p-value

To decide whether to reject a hypothesis one needs the p-value. The procedure for using the p-value in statistical tests is:

  • 1. Definition of the critical p-value or the significance level e.g. 5%
  • 2. Definition of a statistical test procedure e.g. t-tests or correlation analysis
  • 3. Calculation of the test statistic from the sample e.g. the t-value in the t-test
  • 4. Determination of the p-value for the test statistic e.g. p-value for given t in t-test
  • 5. Check whether the p-value is above or below the specified critical p-value e.g. p-value 1% lies below the critical value of 5%

Statistical significance does not show whether an effect is large or practically important. Report the p-value together with the effect estimate and, where possible, a confidence interval. A non-significant result does not prove that there is no effect; it may also reflect limited data or high variability. Because p-values often become smaller as sample size increases, even a very small effect can be statistically significant in a large sample.

If many hypotheses are tested, the chance of obtaining at least one small p-value by chance increases. In that situation, use an appropriate multiple-testing adjustment or clearly state how many tests were performed.


Statistics made easy

  • many illustrative examples
  • ideal for exams and theses
  • statistics made easy on 464 pages
  • 8th revised edition (March 2026)

Only €8.99

Free sample
numiqo

"Super simple written"

"It could not be simpler"

"So many helpful examples"

Cite numiqo: numiqo Team (2026). numiqo: Online Statistics Calculator. numiqo e.U. Graz, Austria. URL https://numiqo.com