Two-way ANOVA

What is a two-way ANOVA?

Two-way (or two-factor) analysis of variance examines how two categorical factors are associated with the mean of a metric dependent variable. It tests the main effect of each factor and the interaction between them using independent observations.

What is a factor?

A factor is, for example, the gender of a person with the characteristics male and female, the form of therapy used for a disease with therapy A, B and C or the field of study with, for example, medicine, business administration, psychology and math.

In analysis of variance, a factor is a categorical variable, and its categories are called levels. ANOVA can test whether the mean of the dependent variable differs between these levels.

For example, you could test whether mean salary differs by gender, whether mean blood pressure differs between therapies, or whether mean study duration differs by field of study. Salary, blood pressure or study duration are then the dependent variables. In all these cases you can test whether the mean of the dependent variable differs between factor levels.

Since you only have one factor in these cases, you would use a one-way analysis of variance. With only two groups, an independent-samples t-test is often used; the corresponding one-way ANOVA gives the same two-sided significance result under the equal-variance model.

Two factors (two-way) ANOVA

Now you may have another categorical variable that you want to include as well. You might be interested in whether:

• in addition to gender, the highest level of education also has an influence on salary.
• besides therapy, gender also has an influence on blood pressure.
• in addition to the field of study, the university attended also has an influence on the duration of studies.

In all three cases you would not have one factor, but two factors each. And since you now have two factors, a two-way analysis of variance can examine them together.

Using the two-way analysis of variance, you can now answer three things:

• Does factor 1 have an effect on the dependent variable?
• Does factor 2 have an effect on the dependent variable?
• Is there an interaction between factor 1 and factor 2?

Here, an "effect" means a difference in population means. A causal interpretation requires a suitable design, such as randomized assignment and control of relevant confounding variables; ANOVA alone does not establish causality.

Therefore, in the case of one-way analysis of variance, we have one factor from which we create the groups. In the case of two-way analysis of variance, the groups result from the combination of the categories of the two factors.

Example two-way ANOVA

Here's an example dataset for a two-way ANOVA in medicine. Let's say we are interested in studying the effect of two factors, "Treatment" and "Gender," on the response variable "Blood Pressure."

In this example, we have two categories of the "Treatment" factor (A and B) and two levels of the "Gender" factor (Male and Female). The "Blood Pressure" measurements are recorded for each participant based on their treatment and gender.

To perform a two-way ANOVA on this dataset, we would test the null hypothesis that there is no main effect of treatment, the null hypothesis that there is no main effect of gender, and the null hypothesis that there is no interaction between treatment and gender.

Hypotheses

Three statements can be tested with the two-way ANOVA, so there are 3 null hypotheses and therefore 3 alternative hypotheses.

Assumptions

For a two-way analysis of variance to be calculated without repeated measures, the following assumptions must be met:

• The dependent variable should be metric, and the independent variables (factors) should be categorical.
• Independence: The observations should be independent. For example, each participant should contribute only one observation to the analysis. Repeated observations from the same participant require a repeated-measures or mixed model.
• Homogeneity: The variances in the groups formed by the combinations of factor levels should be approximately equal. This can be checked with Levene's test.
• Normality: The model residuals within the groups should be approximately normally distributed.
• Replication: To estimate the interaction and error in the usual two-way ANOVA, the dataset needs more than one independent observation in each combination of factor levels.

So the dependent variable could be, for example, salary, blood pressure, and study duration. These are all metric variables. And the independent variable should be nominally or ordinally scaled. For example, gender, highest level of education, or a type of therapy. Note, however, that rank order is not used with ordinal variables, so this information is lost when they are treated simply as categories.

Calculate two-way ANOVA

To calculate a two-way ANOVA, the following formulas are needed. The formulas shown here apply to the balanced example, which has the same number of observations in every combination of factor levels.

Let's say you work in the marketing department of a bank and you want to find out if gender and the fact that a person has studied or not have an influence on their attitude towards retirement planning.

In this example, your two independent variables (factors) are gender (male or female) and study (yes or no). Your dependent variable is attitude toward retirement planning, where 1 means "not important" and 10 means "very important."

After all, is attitude toward retirement planning really a metric variable? Let's just assume that attitude toward retirement planning was measured using a multi-item Likert scale whose combined score is treated as approximately metric. A single Likert-type item is ordinal and should not automatically be treated as metric.

Mean Values

In the first step we calculate the mean values of the individual groups, i.e. of male and not studied, which is 5.8 then of male and studied, which is 5.4, we now do the same for female.

Then we calculate the mean of all male and female and of not studied and studied respectively. Finally, we calculate the overall mean as 5.4.

Sums of Squares

With this, we can now calculate the required sums of squares. SS_tot is the sum of squares of each individual value minus the overall mean.

SS_btw results from the sum of squares of the group means minus the overall mean multiplied by the number of values in the groups.

The sums of squares of the factors SS_A and SS_B result from the sum of squares of the means of the factor levels minus the total mean.

Now we can calculate the sum of squares for the interaction. These are obtained by calculating SS_btw minus SS_A minus SS_B.

Finally, we calculate the sum of squares for the error. This will calculate similar to the total sum of squares, so again we use each individual value. Only in this case, instead of subtracting the overall mean from each value, we subtract the respective group mean from each value.

Degrees of Freedom

The required degrees of freedom are as follows:

Mean Squares or Variance

Together with the sums of squares and the degrees of freedom, the variance can now be calculated:

F-value

And now we can calculate the F-values. These are obtained by dividing the variance of factor A, factor B or the interaction AB by the error variance.

p-value

To calculate the p-value, we need the F-value, the degrees of freedom and the F-distribution. We use the F-distribution p-value calculator numiqo. Of course, you can also just calculate the example completely with numiqo, more about that in the next section.

This gives us a p-value of 0.323 for Factor A, a p-value of 0.686 for Factor B, and a p-value of 0.55 for the interaction. None of these p-values is less than 0.05 and thus we retain the respective null hypotheses.

Calculating two-way ANOVA with numiqo

We take the same example from above. The data is now arranged in the form so that your statistics software can do something with it. In each row is a respondent.

This example consists of only 20 cases, which of course is not much, giving us very low test power, but as an example it should fit.

To calculate a two-way analysis of variance online, simply visit numiqo.com and copy your own data into this table.

Then click on Hypothesis tests. Under this tab you will find a lot of hypothesis tests and depending on which variable you select, you will get an appropriate hypothesis test suggested.

When you copy your data into the table, the variables appear under the table, if the correct scale level is not automatically detected, you can simply change it by clicking on the scale level itself.

We want to know if gender and whether you have studied or not has an impact on your attitude towards retirement planning. So we just click on all three variables.

numiqo will now automatically calculate a two-way analysis of variance without repeated measures. numiqo outputs the three null and the three alternative hypotheses, then the descriptive statistics and the Levene's test of equality of variances. With the Levene's test you can assess whether there is evidence that the group variances differ. Here, the p-value is greater than 0.05, so the test does not detect a violation of equal variances. This does not prove that the variances are equal, especially because Levene's test has low power in small samples.

Next come the results of the two-way ANOVA.

Interpreting two-way ANOVA

The most important in this table are the three marked rows. With these three rows, you can test whether the 3 null hypotheses we made earlier are kept or rejected.

The first row tests your null hypothesis of whether studied or not studied has an effect on attitude towards retirement planning. The second row tests whether gender has an effect on attitude. Finally the third row tests, the interaction between studied and gender.

You can read the p-value in each case right at the last column. Let's say we set the significance level at 5%. If our calculated p-value is less than 0.05, then the null hypothesis is rejected, and if the calculated p-value is greater than 0.05, the null hypothesis is not rejected.

Thus, in this case, we see that all three p-values are greater than 0.05 and thus we cannot reject any of the three null hypotheses.

Therefore, neither whether one has studied nor gender has a statistically significant effect on attitudes toward retirement planning in this sample. There is also no statistically significant interaction between studied and gender. These non-significant results mean that the data do not provide sufficient evidence of these effects; they do not prove that the effects are exactly zero.

The p-values do not describe how large the effects are. An effect-size measure such as partial eta squared should therefore be reported and interpreted in the context of the study, ideally together with confidence intervals.

If you don't know exactly how to interpret the results, you can also just click on Summary in Words. In addition, it is important to check in advance whether the assumptions for the analysis of variance are met at all.

Interaction effect

But what exactly does interaction mean? Let us first have a look at this diagram.

The dependent variable is plotted on the y axis, in our example: the attitude towards retirement provision. On the x axis, one of the two factors is plotted, let's just take gender. The other factor is represented by lines with different colors. Green is studied and red is not studied.

The endpoints of the lines are the mean values of the groups, e.g. male and not studied.

In this diagram, the sample means are higher for females than for males and higher for people who studied than for people who did not. The plot shows descriptive patterns; the ANOVA is needed to assess whether there is statistical evidence for the corresponding population effects.

But now finally to the interaction effects, for that we compare these two graphs.

In the first case, we said there is no interaction effect. If a person has studied, he has a value that is, say, 1.5 higher than a person who has not studied.

This increase of 1.5 is independent of whether the person is male or female.

It is different in this case, here studied persons also have a higher value, but how much higher the value is depends on whether one is male or female. If I am male, there is a difference of, let's say for example 0.5 and if I am female, there is a difference of 3.5.

So in this case the graph clearly illustrates an interaction between gender and study: the difference associated with studying depends on whether the person is male or female.

In this case, we do have an interaction effect, but the direction still remains the same. So females have higher scores than males and studied have higher scores than non-studied.

When an interaction is statistically significant, interpret it before the main effects. Simple-effects comparisons can then examine one factor separately at each level of the other factor. If a factor has more than two levels, adjusted pairwise comparisons may also be needed to identify which levels differ while controlling for multiple testing.