ROC Curve

What is a ROC curve?

A ROC curve is a graphical representation of the performance of a binary classification model across all classification thresholds. Here, ROC stands for Receiver Operating Characteristic.

We want to classify, based on a screening, whether a person has cancer or not.

This classification uses a blood value where higher values indicate cancer. The question is which value we choose as the classification threshold. From which value do we predict disease?

For this, we obtain data from 10 people on their blood value and whether the disease is present.

We could choose a classification threshold of 45, for example. In this case, of the five people with the disease, we would correctly classify four as "diseased" and one incorrectly as "healthy." Thus, we correctly classified 4 out of 5 as "diseased."

This value is called True Positive Rate (TPR) and is equal to the sensitivity.

On the other hand, out of the five healthy individuals, we misclassify two as "diseased" and correctly classify three as "healthy." So, we misclassified 2 out of 5 as "diseased." This value is called the False Positive Rate (FPR).

So for a threshold of 45 we get a True Positive Rate of 4/5, i.e. 0.8, and a False Positive Rate of 2/5, i.e. 0.4.

True Positive Rate and False Positive Rate

The True Positive Rate (TPR) is calculated with this equation:

The True Positive Rate is equal to the true positives divided by the true positives plus the false negatives. The true positives are those correctly classified as "diseased" and the false negatives are those incorrectly classified as "healthy."

The False Positive Rate (FPR) is obtained using this equation:

The False Positive Rate is equal to the false positives divided by the false positives plus the true negatives. The false positives are the healthy individuals misclassified as "diseased" and the true negatives are the individuals correctly classified as "healthy."

How to plot the ROC Curve

We can now calculate the True Positive Rate and False Positive Rate for each threshold. These two values are plotted on the ROC curve. The True Positive Rate is plotted on the y-axis and the False Positive Rate on the x-axis.

Now let's plot the complete ROC curve for our example!

If we choose a very small threshold value, i.e., pushing it all the way to the left, we correctly classify all five diseased individuals. Our True Positive Rate is thus 5 out of 5, i.e. 1.

In the same way, we also misclassify all five healthy people as "diseased." Our False Positive Rate is therefore 5 out of 5, i.e. 1.

This gives us the first point:

Now we can increase the threshold. Here we still classify all five diseased people correctly as "diseased." So we still have a True Positive Rate of 5/5. However, of the five healthy individuals, we now only misclassify 4 out of 5 as "diseased." So we have 4 out of 5, or 0.8.

At the next threshold, we still have a True Positive Rate of 1. All five diseased are correctly classified. The False Positive Rate takes the value of 3/5, so 0.6.

At the next threshold, for the first time, a diseased person is misclassified as "healthy". We therefore obtain a True Positive Rate of 4/5, i.e. 0.8, and a False Positive Rate of 3/5, i.e. 0.6.

We can do this for all other thresholds, finishing the ROC curve. At the marked point below, for example, 80% of the diseased people were correctly classified as "diseased" and 20% of the healthy people were incorrectly classified as "diseased."

AUC value

Using the ROC curve, we can also compare different classification methods. A classifier is better when the curve is higher and closer to the top-left corner. Therefore, the larger the area under the curve, the better the classifier. This area, the Area under the Curve, is reflected by the AUC value.

The AUC value varies between 0 and 1. The larger the value, the better the classifier.

ROC curve and logistic regression

How does this relate to logistic regression? We could, for example, create a new classifier using logistic regression. Here we could use, in addition to the blood value, a person's age and gender.

In logistic regression, the estimated value is the probability that a particular person has the disease.

Very often, 50% is used as the threshold to classify whether a person is "diseased" or not. But this does not have to be the case. Any threshold can be used.

Therefore, we can also create a ROC curve by varying the threshold in logistic regression.

Create ROC curve with numiqo

Of course, we can easily output a ROC curve online with numiqo. To do this, we simply copy our data into the table and click on ROC Calculator. Alternatively, you can also create an ROC curve in the Regression Calculator in Logistic Regression.

We simply select the two variables Diseased and Blood Value and specify what we consider a positive event, in our case the answer yes. Now we get the ROC curve. In the table below the ROC curve, you can see the threshold value for each point on the curve.