R - Random Forest

R - Modern Random Forest with tidymodels

A Random Forest is a powerful machine learning algorithm that builds a multitude of decision trees and aggregates their predictions. It’s an ensemble method that generally provides high accuracy, robustness against overfitting, and an estimate of feature importance.

While the randomForest package is functional, the modern and recommended approach for modeling in R is the tidymodels framework. tidymodels provides a consistent, tidy, and comprehensive ecosystem for the entire modeling process, from data splitting to model evaluation.

The tidymodels Workflow

Our process will follow these key steps:

1. Setup: Install and load the necessary packages.
2. Data Splitting: Split our data into training and testing sets for robust evaluation.
3. Model Specification: Define the random forest model and its engine (the underlying package that does the computation).
4. Workflow: Bundle the model specification into a workflow object.
5. Training: Fit the workflow to the training data.
6. Evaluation: Predict on the test set and evaluate performance using metrics like accuracy and a confusion matrix.
7. Feature Importance: Determine which predictors were most important for the model’s decisions.

Step 1: Setup and Environment

First, install the tidymodels meta-package and party (for the dataset). We’ll also use ranger as our model engine because it’s typically faster than the older randomForest package, and vip for plotting variable importance.

# Run these lines in your console once to install the packages
# install.packages("tidymodels")
# install.packages("party") # For the readingSkills dataset
# install.packages("ranger") # A fast random forest implementation
# install.packages("vip") # For variable importance plots

library(tidymodels)
library(party)
library(dplyr) # For data manipulation

Step 2: Load and Split the Data

We’ll use the readingSkills dataset, which contains information about individuals’ reading scores. Our goal is to predict if a person is a nativeSpeaker based on their age, shoe size, and score. A crucial first step is splitting the data to prevent overfitting and to get an unbiased estimate of our model’s performance on new data.

# Load the data
data("readingSkills")

# Glimpse the data structure
glimpse(readingSkills)

# Set a seed for reproducibility of the random split
set.seed(123)

# Create a data split object (e.g., 80% for training, 20% for testing)
rs_split <- initial_split(readingSkills, prop = 0.80, strata = nativeSpeaker)

# Create the training and testing data frames
rs_train <- training(rs_split)
rs_test  <- testing(rs_split)

Step 3: Model Specification and Workflow

Next, we define our model. We specify that we want a random forest (rand_forest) for a classification task and set ranger as the engine. We then bundle this into a workflow for easy management.

# Specify a random forest model for classification
# We set importance = "impurity" to be able to access it later
rf_spec <- rand_forest(trees = 1000) %>%
  set_engine("ranger", importance = "impurity") %>%
  set_mode("classification")

# Create a workflow
rf_workflow <- workflow() %>%
  add_model(rf_spec) %>%
  add_formula(nativeSpeaker ~ .)

print(rf_workflow)

Step 4 & 5: Train the Model and Make Predictions

Now we can train (fit) our workflow using the training data.

# Train the workflow on the training data
rf_fit <- fit(rf_workflow, data = rs_train)

print(rf_fit)

Step 6: Evaluate Model Performance

How well did our model do? We use the test set—data the model has never seen—to find out. We’ll generate a confusion matrix and calculate accuracy.

# Predict on the test set and combine with true values
predictions <- predict(rf_fit, rs_test) %>%
  bind_cols(rs_test)

# Calculate overall accuracy
accuracy(predictions, truth = nativeSpeaker, estimate = .pred_class)

# Generate a confusion matrix
conf_mat(predictions, truth = nativeSpeaker, estimate = .pred_class)

This produces the following results (your exact numbers may vary slightly):

# A tibble: 1 × 3
  .metric  .estimator .estimate
  <chr>    <chr>          <dbl>
1 accuracy binary         0.975

          Truth
Prediction no yes
       no  19   0
       yes  1  20

Conclusion: Our model achieved 97.5% accuracy on the test set. The confusion matrix shows it made only one error, misclassifying a ‘yes’ as a ‘no’. This is a very strong result.

Step 7: Analyze Feature Importance

Finally, let’s see which variables the model found most useful. We’ll use the vip package for a clear visualization.

# Extract the fitted model object from the workflow
final_model <- extract_fit_engine(rf_fit)

# Create and print the variable importance plot
importance_plot <- vip(final_model)
print(importance_plot)

The plot will show that score is by far the most important predictor, followed by shoeSize and age. This gives us actionable insight into the factors driving the predictions.