Correlation Analysis

What is a Correlation Analysis?

Correlation analysis is a statistical method used to evaluate the relationship between two variables, such as the association between body size and shoe size.

The strength of this relationship is measured by the correlation coefficient, which ranges from -1 to +1. A coefficient close to +1 indicates a strong positive correlation, while a value near -1 signifies a strong negative correlation. Values around zero suggest little to no association of the type measured by the chosen coefficient. For example, Pearson's coefficient measures linear association and can be close to zero even when a strong nonlinear relationship exists. Correlation analysis can thus be used to describe the strength and direction of the measured association.

Correlation vs. Causality

If correlation analysis reveals a relationship between two variables, it is possible to further investigate whether one variable can be used to predict the other (e.g. in a regression model). For instance, if a correlation is found, one could examine whether the age at which a child first speaks sentences can be used to predict their future academic success through linear regression analysis.

However, caution is necessary! Correlations do not imply causation. Any identified correlations should be examined in greater detail and not immediately interpreted as causal relationships, even if a connection seems obvious.

In some cases, the nature or timing of the variables can rule out one causal direction. For example, salary cannot change a person's age. Even then, a correlation between age and salary does not by itself prove that age causes a change in salary; other variables may explain some or all of the association.

Interpreting Correlation

With the help of correlation analysis, two statements can be made:

• one about the direction
• and one about the strength

about an association: its direction and its strength. The exact type of association depends on the coefficient. Pearson's coefficient measures linear association between metric variables, whereas Spearman's coefficient measures monotonic association using ranks and can be used with ordinal variables.

Scatter Plot and Correlation

Just as important as the consideration of the correlation coefficient is the graphical consideration of the correlation of two variables in a scatter diagram.

The scatter plot gives you a rough estimate of whether there is a correlation, whether it is linear or nonlinear, and whether there are outliers.

Test Correlation for Significance

If there is a correlation in the sample, it is still necessary to test whether there is enough evidence that the correlation also exists in the population. Thus, the question arises when a correlation coefficient can be considered statistically significant.

The significance of a Pearson correlation coefficient can be tested using a t-test. As a rule, it is tested whether the population correlation coefficient is different from zero. A zero Pearson correlation means no linear association, but it does not necessarily mean that the variables are independent. In this case, the null hypothesis is that the population Pearson correlation is zero. The alternative hypothesis states that it is not zero.

As with any other hypothesis test, the significance level is first set, usually at 5%. If the calculated p-value is below 5 %, the null hypothesis is rejected. This provides evidence of a linear association in the population, but it does not show that the relationship is causal or indicate whether it is practically important.

The t-value for testing the hypothesis is given by

where n is the sample size and r is the determined correlation in the sample. The corresponding p-value can be easily calculated in the correlation calculator on numiqo.com,

Pearson Correlation Analysis

With the Pearson correlation analysis you get a statement about the linear correlation between metric scaled variables. The respective covariance is used for the calculation. The covariance gives a positive value if there is a positive correlation between the variables and a negative value if there is a negative correlation. The covariance is calculated as:

However, the covariance is not standardized and can assume values between plus and minus infinity. This makes it difficult to compare the strength of relationships between different variables. For this reason, the correlation coefficient, also called product-moment correlation coefficient, is calculated. The correlation coefficient is obtained by normalizing the covariance. For this normalization, the variances of the two variables involved are used and the correlation coefficient is calculated as

The Pearson correlation coefficient can now take values between -1 and +1 and can be interpreted as follows

• The value +1 means that there is an entirely positive linear relationship (the more, the more).
• The value -1 indicates that an entirely negative linear relationship exists (the more, the less).
• A value of 0 means that there is no linear relationship. A nonlinear relationship may still exist.

Now finally the strength of the relationship can be interpreted. This can be illustrated by the following table:

To check in advance whether a linear relationship exists, scatter plots should be considered. This way, the respective relationship between the variables can also be checked visually. The Pearson correlation is only useful and purposeful if linear relationships are present.

Pearson Correlation assumptions

Pearson's coefficient describes the linear relationship between two metric variables. The paired observations should be independent, and a scatter plot should be checked for nonlinearity and influential outliers. For the usual t-test and confidence interval, the pairs should come from an approximately bivariate normal population. Normality can be assessed analytically or graphically with Q-Q plots. If the relationship is monotonic but Pearson's assumptions are not appropriate, the Spearman correlation may be a useful alternative.

Spearman Rank Correlation

Spearman correlation analysis is used to calculate the relationship between two variables that have ordinal level of measurement. Spearman rank correlation measures the strength and direction of a monotonic association using ranks. It is often used for ordinal data or when a monotonic relationship is not well described by Pearson's correlation. It is also referred to as "Spearman correlation" or "Spearman's rho."

The questions that can be treated by Spearman rank correlation are similar to those of the Pearson correlation coefficient, i.e., "Is there an association between two variables or characteristics?" For example: "Is there an association between age and religiosity in the French population?"

The calculation of the rank correlation is based on the ranking system of the data series. This means that the measured values are not used for the calculation, but are transformed into ranks. The test is then performed using these ranks.

For the rank correlation coefficient ρ, values between -1 and 1 are possible. A value less than zero (ρ < 0) indicates a negative monotonic association, while a value greater than zero (ρ > 0) indicates a positive monotonic association. A value of zero (ρ = 0) indicates no monotonic association, although another type of relationship may still exist. The strength can be classified using the following rough guide:

Point Biserial Correlation

The point-biserial correlation is used when one of the variables is dichotomous, e.g. studied and not studied, and the other has metric scale level, e.g. salary.

The calculation of a point biserial correlation is the same as the calculation of the Pearson correlation. To calculate it, one of the two categories of the dichotomous variable is coded as 0 and the other as 1.

Calculate Correlation Analysis with numiqo

A student wants to know if there is a correlation between the height and weight of the participants in the statistics course. For this purpose, the student drew a sample, which is described in the table below.

To analyze the linear relationships by means of a correlation analysis, you can calculate a correlation with numiqo. First copy the table above into the statistics calculator. Then click on "Correlation" and select the two variables from the example. Finally you will get the following results.

First, you will get the null and the alternative hypothesis. The null hypothesis is: "There is no correlation between height and weight". Then you get the correlation coefficient and the p-value. If you click on Summary in words, you will get the following interpretation:

Directional (one-sided) Correlation Hypothesis

Of course, in numiqo you can also choose to calculate a directional hypothesis.

A one-sided hypothesis must be specified before examining the data. Here, the alternative hypothesis is that height and weight are positively correlated. numiqo calculates the corresponding one-sided p-value and also checks whether the sample correlation is in the hypothesized direction. The summary in words then looks like this: