Random Forest

With numiqo you can create a Random Forest model from one dependent variable and two or more predictor variables. If the dependent variable is categorical, numiqo runs Random Forest classification. If the dependent variable is metric, numiqo runs Random Forest regression.

To calculate a Random Forest online, open Predictive Analytics, select Random Forest, choose the dependent variable, and select the independent variables that should be used as predictors.

What is a Random Forest?

A Random Forest is an ensemble method. Instead of using one decision tree, it builds many decision trees and combines their predictions. Each tree is trained on a bootstrap sample of the data, and at each split only a random subset of predictors is considered. This makes the model less dependent on one unstable tree and usually improves predictive performance.

For classification, each tree votes for a category of the dependent variable and the forest predicts the category with the most votes. For regression, each tree predicts a numeric value and the forest uses the average prediction across trees.

How the Random Forest calculator works

The Random Forest model follows a CART-style approach. For every bootstrap sample, it grows a binary tree. Classification splits are evaluated with Gini impurity. Regression splits are evaluated with squared error. The final model combines all trees by majority vote for classification or by averaging predictions for regression.

1. Bootstrap samples: Each tree is trained on a random sample drawn with replacement from the original rows.
2. Random predictor selection: At each node, only a subset of predictors is considered for the split. By default, the calculator uses the square root of the number of selected predictors.
3. Binary splits: Categorical predictors are split by separating one category from the remaining categories. Metric predictors are split with threshold rules.
4. Combined prediction: Classification uses majority vote. Regression averages the numeric predictions from the trees.
5. Out-of-bag validation: Rows not used in a tree's bootstrap sample are predicted by that tree. These out-of-bag predictions estimate model performance without requiring a separate test set.

Random Forest results

The Random Forest output includes a method table, out-of-bag performance, and relative variable importance. Classification models also include a confusion matrix. Regression models report RMSE, MAE, and R-squared.

Out-of-bag accuracy

Out-of-bag accuracy is the percentage of correctly classified rows using only trees for which that row was not included in the bootstrap sample. It is a practical internal validation measure and is commonly used with Random Forest models.

Regression performance

For Random Forest regression, the out-of-bag table reports RMSE, MAE, and R-squared. RMSE and MAE describe the typical prediction error in the units of the dependent variable. R-squared describes how much out-of-bag variance is explained by the predictions.

Confusion matrix

The confusion matrix compares the observed category with the predicted category. The rows show the actual categories, and the columns show the predicted categories. The correct percentage for each row shows how often observations from that actual category were classified correctly.

Variable importance

Variable importance summarizes how much each predictor improves the tree splits across the forest. A higher relative importance means that the variable contributed more to improving the splits across the forest. Importance is useful for ranking predictors, but it should not be interpreted as proof of causality.

Random Forest vs. Decision Tree

A single decision tree is easier to interpret because the complete tree can be inspected. A Random Forest is usually stronger for prediction because it averages many trees, but it summarizes the model with performance metrics and variable importance instead of one simple tree diagram. For categorical dependent variables, CHAID is also available as an alternative tree method.

When to use Random Forest

Use Random Forest classification when the dependent variable is categorical and the goal is to predict group membership. Use Random Forest regression when the dependent variable is metric and the goal is to predict a numeric value. Common applications include customer churn prediction, risk classification, quality inspection, demand prediction, and customer segmentation.

Limitations

Rows with missing values in the selected variables are excluded from the calculation. The current implementation uses CART-style binary splits and internal out-of-bag validation. It does not yet include partial dependence plots, automated hyperparameter search, or advanced missing-value penalty options.