Chi-Square Test

The Chi-square test is a hypothesis test used to determine whether there is a relationship between two categorical variables.

What are categorical variables? Categorical variables are, for example, a person's gender, preferred newspaper, frequency of television viewing, or their highest level of education. To test whether there is a relationship between two categorical variables, you can use a chi-square test of independence.

Let's say we want to investigate whether there is a connection between gender and the highest level of education. To do this, we create a questionnaire in which the participants tick their gender and what their highest educational level is. The result of the survey is then displayed in a contingency table.

The chi-square test is used to investigate whether there is a relationship between gender and the highest level of education.

Tip: With numiqo you can calculate the chi-square test online. Simply visit the Chi-Square Test Calculator.

Applications of the Chi-Square Test

There are various applications of the chi-square test. It can be used to answer the following questions:

1) Independence test

Are two categorical variables independent of each other? For example, does gender have an impact on whether a person has a Netflix subscription or not?

2) Distribution test

Do the observed frequencies of one categorical variable match a specified distribution? One question could be whether subscriptions to Netflix, Amazon, and Disney follow specified market shares.

3) Homogeneity Test

Do two or more populations have the same distribution of a categorical variable? One question could be whether the subscription frequencies of Netflix, Amazon, and Disney differ among age groups.

Calculate chi-square

The chi-square value is calculated via:

To clarify the calculation of the chi-square value, we refer to the following case: for variables one and two with category A and B, an observation was made or a sample exists. Now we want to check whether the frequencies from the sample correspond to the expected frequencies from the population.

With the upper equation you can now calculate chi-squared:

After calculating chi-squared, the number of degrees of freedom df is needed. This is given by

with

• p: number of lines
• q: number of columns

From the table of the chi-squared distribution

one can now read the critical chi-square value. For a significance level of 5% and a df of 1, this results in 3.841. Since the calculated chi-square value is smaller, the result is not statistically significant.

As an assumption for the usual chi-square approximation, expected frequencies should generally be 5 or more in at least 80% of cells, with none below 1.

Chi-Square Test of Independence

The Chi-Square Test of Independence is used when two categorical variables are to be tested for independence. The aim is to analyze whether the distribution of one variable differs across the categories of the other. The test detects association, but it does not by itself establish that one variable causes the other.

For example, does gender have an influence on whether a person has a Netflix subscription or not? For the two variables gender ("male", "female") and has Netflix subscription ("yes", "no"), it is tested whether they are independent. If this is not the case, there is a relationship between the variables.

The research question that can be answered with the Chi-square test is: Are the characteristics of gender and ownership of a Netflix subscription independent of each other?

In order to calculate the chi-square, an observed and an expected frequency must be given. In the independence test, the expected frequency is the one that results when both variables are independent. If two variables are independent, the expected frequencies of the individual cells are obtained with

where i and j are the rows and columns of the table respectively.

For the fictitious Netflix example, the following tables could be used. On the left is the table with the frequencies observed in the sample, and on the right is the table that would result if perfect independence existed.

The Chi-square is then calculated as

From the Chi-square table you can now read the critical value again and compare it with the result.

The assumptions for the chi-square independence test are that observations are independent, categories are mutually exclusive, and the data are counts. For the usual chi-square approximation, expected frequencies should generally be at least 5 in 80% of cells and none should be below 1.

Chi-square Distribution Test

If a variable is present with two or more categories, the differences in the frequency of the individual values can be examined.

The Chi-square distribution test, or Goodness-of-fit test, checks whether the frequencies of the individual characteristic values in the sample correspond to the frequencies of a defined distribution. In most cases, this defined distribution corresponds to that of the population. In this case, it is tested whether the sample comes from the respective population.

For market researchers it could be of interest whether there is a difference in the market penetration of the three video streaming services Netflix, Amazon and Disney between Berlin and the whole of Germany. The expected frequency is then the distribution of streaming services throughout Germany and the observed frequency results from a survey in Berlin. In the following tables the fictitious results are shown:

The Chi-square then results in

Chi-square Homogeneity Test

The Chi-square homogeneity test can be used to check whether two or more samples come from the same population. One question could be whether the subscription frequency of three video streaming services Netflix, Amazon and Disney differ in different age groups. As a fictitious example, a survey is made in three age groups with the following result:

As with the Chi-square independence test, this result is compared with the expected table assuming independence of "Streaming providers" and "age".

Effect size in the Chi-square test

So far we only know whether we can reject the null hypothesis or not, but it is very often of great interest to know how strong the relationship between the two variables is. This can be answered with the help of the effect size.

For a chi-square test of association, Cramer's V can be used as an effect size. The often-used benchmarks of 0.1, 0.3, and 0.5 for small, medium, and large effects apply most directly to a 2 × 2 table; interpretation should also consider the table size and the subject area. numiqo calculates Cramer's V for you.

A significant chi-square result indicates that an association exists, but not which cells are responsible for it. Comparing observed and expected counts, preferably with standardized residuals, helps identify the categories that contribute most to the result.

Effect size vs. p-value

Please note that the p-value does not tell you anything about the strength of the association or the effect and depends on the sample size! The following points should therefore be considered:

• If there is an association in the population, a larger sample generally provides more power to detect it.
• If the sample is very large, very small associations can also be detected in the population.
• These small associations may not be practically relevant.

Therefore, if a small sample and a large sample show the same estimated effect, their p-values can still differ. All else being equal, the larger sample usually produces the smaller p-value, so even a very small association may be statistically significant.

This is where the effect size plays an important role. With the effect size in the Chi-square test, differences can be made comparable across several studies.

Example Chi-Square Test (Independence Test)

As an example of a chi-square test where independence is tested, we consider the use of umbrellas. On a rainy day we counted how many women and how many men come to university with an umbrella.

This is how it works in the online statistics calculator: After you have copied the above table into the hypothesis test calculator, you can calculate the chi-square test. To do this, simply click on the two variables Gender and Umbrella. As a result, you will get the (1) contingency table, the (2) expected frequency for perfectly independent variables and the (3) chi-square test

Expected frequencies for perfectly independent variables:

With a significance level of 5% and a degree of freedom of 1, the table of chi-square values gives a critical value of 3.841. Since the calculated chi-square value is smaller than the critical value, the result is not statistically significant and the null hypothesis is not rejected. These data therefore do not provide evidence of an association between gender and umbrella use.

Distribution test

In one district of Vienna, the party membership of 22 persons was recorded. Now it is to be examined whether the residents of the district (random sample) have the same voting behaviour as the residents of the entire city of Vienna (population).

To calculate the chi-square test for the example, simply copy the upper table into the Hypothesis Test Calculator.

Party A has a 40% share in Vienna and party C has 35%. You will therefore now receive the following results:

If the significance level is set at 0.05, the p-value calculated at 0.876 is greater than the significance level. Thus, the null hypothesis is not rejected. These data do not provide evidence that the district's voting behavior differs from that of the entire city of Vienna.