Random Forest in R Tutorial: Algorithm with Example
Evelyn Clarke
⚡ Smart Summary
Random Forest in R builds hundreds of decision trees on bootstrapped samples and averages their votes into one robust prediction. This walkthrough tunes mtry, maxnodes, and ntree with caret and evaluates the final model on the Titanic survival data.
What is Random Forest in R?
Random forests are based on a simple idea: ‘the wisdom of the crowd’. Aggregate of the results of multiple predictors gives a better prediction than the best individual predictor. A group of predictors is called an ensemble. Thus, this technique is called Ensemble Learning.
In an earlier tutorial, you learned how to use decision trees to make a binary prediction. To improve our technique, we can train a group of Decision Tree classifiers, each on a different random subset of the train set. To make a prediction, you collect the predictions of all the individual trees and return the class that receives the most votes. This technique is called Random Forest.
Before writing any code, it helps to see exactly how the forest is assembled from those individual trees.
How Does Random Forest Work in R?
Understanding the mechanics makes every tuning parameter easier to reason about. A random forest is built in four stages.
- Bootstrap sampling. The algorithm draws a random sample of rows with replacement from the training set for every tree. Roughly one third of the rows are left out of each sample; these are the out-of-bag (OOB) observations.
- Random feature selection. At every split, only a random subset of predictors is considered. That subset size is the mtry parameter. Restricting the choice is what stops all the trees from looking alike.
- Unpruned tree growth. Each tree grows until it hits a stopping rule such as maxnodes or nodesize. Individual trees are deliberately allowed to overfit, because their errors are uncorrelated.
- Aggregation. For classification the forest returns the class with the most votes; for regression it returns the average prediction. This averaging step is what the term bagging (bootstrap aggregating) describes.
Row sampling plus column sampling is the whole trick. A single deep tree has low bias and very high variance; averaging hundreds of them keeps the low bias while driving the variance down.
Out-of-bag error. Because every observation is excluded from about a third of the trees, R can score each row using only the trees that never saw it. The resulting OOB error is a free, built-in validation estimate printed by randomForest():
rf_oob <- randomForest(survived~., data = data_train, ntree = 800, mtry = 4) print(rf_oob) # reports the OOB estimate of error rate
The OOB error is a convenient quick check, but this tutorial uses ten-fold cross-validation through caret so every tuning grid is compared on identical folds.
Random Forest vs Decision Tree in R
A random forest is an ensemble of the same decision trees covered in the previous tutorial, so the differences are worth stating plainly before you choose between them.
| Criteria | Decision Tree | Random Forest |
|---|---|---|
| Structure | One tree | Hundreds of trees combined by voting |
| Variance | High, very sensitive to the training sample | Low, averaging cancels individual errors |
| Overfitting risk | High unless pruned | Low even with unpruned trees |
| Interpretability | Fully readable as a flowchart | Only variable importance is readable |
| Training cost | Very fast | Proportional to ntree |
| Built-in validation | None | Out-of-bag error estimate |
| R function | rpart() | randomForest() |
Choose a single tree when you must explain the decision path to a non-technical audience. Choose a forest whenever predictive accuracy matters more than a readable diagram.
Advantages and Disadvantages of Random Forest
Knowing where the algorithm is strong and weak tells you when the tuning effort ahead is worth spending.
Advantages
- Accuracy without pruning: averaging many uncorrelated trees delivers strong results with very little manual tuning.
- Resistant to overfitting: adding more trees never increases the generalisation error, so ntree can be raised safely.
- Handles mixed data: numeric and factor predictors work side by side, and scaling is unnecessary.
- Free validation and ranking: the OOB error and varImp() come at no extra computational cost.
Disadvantages
- Opaque predictions: you cannot trace a single decision path, which matters in regulated environments.
- Slow on large forests: training and prediction time scale linearly with the number of trees.
- Biased importance scores: categorical variables with many levels can look more important than they truly are.
- Weak extrapolation: for regression the forest can never predict outside the range seen in training.
With the theory and the trade-offs settled, the next six steps build, tune, and evaluate a random forest end to end on the Titanic survival dataset.
Step 1) Import the data
To make sure you have the same dataset as in the tutorial for decision trees, the train set and the test set are hosted online. You can import them without making any change.
library(dplyr) data_train <- read.csv("https://raw.githubusercontent.com/guru99-edu/R-Programming/master/train.csv") glimpse(data_train) data_test <- read.csv("https://raw.githubusercontent.com/guru99-edu/R-Programming/master/test.csv") glimpse(data_test)
Step 2) Train the model
One way to evaluate the performance of a model is to train it on a number of different smaller datasets and evaluate them over the other smaller testing set. This is called k-fold cross-validation. R has a function that randomly splits the data into k subsets of almost the same size. For example, if k = 10, the model is trained on nine folds and evaluated on the remaining fold. This process is repeated until all the subsets have been evaluated. This technique is widely used for model selection, especially when the model has parameters to tune.
Now that we have a way to evaluate our model, we need to decide which parameters generalize best to unseen data.
Random forest chooses a random subset of features and builds many Decision Trees. The model averages out all the predictions of the Decisions trees.
Random forest has some parameters that can be changed to improve the generalization of the prediction. You will use the randomForest() function to train the model.
The syntax for randomForest() is:
randomForest(formula, ntree=n, mtry=FALSE, maxnodes = NULL) Arguments: - Formula: Formula of the fitted model - ntree: number of trees in the forest - mtry: Number of candidate variables drawn at each split. By default, it is the square root of the number of predictors for classification. - maxnodes: Set the maximum number of terminal nodes each tree can have - importance=TRUE: Whether independent variables importance in the random forest be assessed
Note: Random forest can be trained on more parameters. You can refer to the vignette to see the different parameters.
Tuning a model is tedious work. There are many possible combinations of parameters. You don’t necessarily have the time to try all of them. A good alternative is to let the machine find the best combination for you. There are two methods available:
- Random Search
- Grid Search
Both methods are defined below, but this tutorial trains the model using grid search.
Grid Search definition
The grid search method is simple, the model will be evaluated over all the combination you pass in the function, using cross-validation.
For instance, you want to try the model with 10, 20, 30 number of trees and each tree will be tested over a number of mtry equals to 1, 2, 3, 4, 5. Then the machine will test 15 different models:
.mtry ntrees 1 1 10 2 2 10 3 3 10 4 4 10 5 5 10 6 1 20 7 2 20 8 3 20 9 4 20 10 5 20 11 1 30 12 2 30 13 3 30 14 4 30 15 5 30
The algorithm will evaluate:
randomForest(formula, ntree=10, mtry=1) randomForest(formula, ntree=10, mtry=2) randomForest(formula, ntree=10, mtry=3) randomForest(formula, ntree=20, mtry=2) ...
Each combination is evaluated with cross-validation. The shortcoming of grid search is the number of experiments: it grows explosively when the number of combinations is high. To overcome this issue, you can use random search.
Random Search definition
The big difference between random search and grid search is, random search will not evaluate all the combination of hyperparameter in the searching space. Instead, it will randomly choose combination at every iteration. The advantage is a much lower computational cost.
Set the control parameter
You will proceed as follow to construct and evaluate the model:
- Evaluate the model with the default setting
- Find the best number of mtry
- Find the best number of maxnodes
- Find the best number of ntrees
- Evaluate the model on the test dataset
Before you begin with the parameters exploration, you need to install two libraries.
- caret: R machine learning library. If you have install R with r-essential. It is already in the library
- Anaconda: conda install -c r r-caret
- e1071: R machine learning library.
- Anaconda: conda install -c r r-e1071
You can import them along with randomForest:
library(randomForest) library(caret) library(e1071)
Default setting
K-fold cross validation is controlled by the trainControl() function
trainControl(method = "cv", number = n, search ="grid") arguments - method = "cv": The method used to resample the dataset. - number = n: Number of folds to create - search = "grid": Use the grid search method. For the randomized method, use "random" Note: You can refer to the vignette to see the other arguments of the function.
You can try to run the model with the default parameters and see the accuracy score.
Note: You will use the same controls during all the tutorial.
# Define the control trControl <- trainControl(method = "cv", number = 10, search = "grid")
You will use caret library to evaluate your model. The library has one function called train() to evaluate almost all machine learning algorithm. Say differently, you can use this function to train other algorithms.
The basic syntax is:
train(formula, df, method = "rf", metric= "Accuracy", trControl = trainControl(), tuneGrid = NULL) argument - `formula`: Define the formula of the algorithm - `method`: Define which model to train. Note, at the end of the tutorial, there is a list of all the models that can be trained - `metric` = "Accuracy": Define how to select the optimal model - `trControl = trainControl()`: Define the control parameters - `tuneGrid = NULL`: Return a data frame with all the possible combination
Let’s build the model with the default values.
set.seed(1234) # Run the model rf_default <- train(survived~., data = data_train, method = "rf", metric = "Accuracy", trControl = trControl) # Print the results print(rf_default)
Code Explanation
- trainControl(method=”cv”, number=10, search=”grid”): Evaluate the model with a grid search over 10 folds
- train(…): Train a random forest model. Best model is chosen with the accuracy measure.
Output:
## Random Forest ## ## 836 samples ## 7 predictor ## 2 classes: 'No', 'Yes' ## ## No pre-processing ## Resampling: Cross-Validated (10 fold) ## Summary of sample sizes: 753, 752, 753, 752, 752, 752, ... ## Resampling results across tuning parameters: ## ## mtry Accuracy Kappa ## 2 0.7919248 0.5536486 ## 6 0.7811245 0.5391611 ## 10 0.7572002 0.4939620 ## ## Accuracy was used to select the optimal model using the largest value. ## The final value used for the model was mtry = 2.
The algorithm uses 500 trees and tested three different values of mtry: 2, 6, 10.
The final value used for the model was mtry = 2, with a cross-validated accuracy of 0.792. Let’s try to get a higher score.
Search the best mtry
You can test the model with values of mtry from 1 to 10
set.seed(1234) tuneGrid <- expand.grid(.mtry = c(1: 10)) rf_mtry <- train(survived~., data = data_train, method = "rf", metric = "Accuracy", tuneGrid = tuneGrid, trControl = trControl, importance = TRUE, nodesize = 14, ntree = 300) print(rf_mtry)
Code Explanation
- tuneGrid <- expand.grid(.mtry = c(1:10)): Construct a vector with values from 1 to 10
Output:
## Random Forest ## ## 836 samples ## 7 predictor ## 2 classes: 'No', 'Yes' ## ## No pre-processing ## Resampling: Cross-Validated (10 fold) ## Summary of sample sizes: 753, 752, 753, 752, 752, 752, ... ## Resampling results across tuning parameters: ## ## mtry Accuracy Kappa ## 1 0.7572576 0.4647368 ## 2 0.7979346 0.5662364 ## 3 0.8075158 0.5884815 ## 4 0.8110729 0.5970664 ## 5 0.8074727 0.5900030 ## 6 0.8099111 0.5949342 ## 7 0.8050918 0.5866415 ## 8 0.8050918 0.5855399 ## 9 0.8050631 0.5855035 ## 10 0.7978916 0.5707336 ## ## Accuracy was used to select the optimal model using the largest value. ## The final value used for the model was mtry = 4.
The best value of mtry is 4. It is stored in:
rf_mtry$bestTune$mtry
You can store it and use it when you need to tune the other parameters.
max(rf_mtry$results$Accuracy)
Output:
## [1] 0.8110729
best_mtry <- rf_mtry$bestTune$mtry best_mtry
Output:
## [1] 4
Step 3) Search the best maxnodes
You need to create a loop to evaluate the different values of maxnodes. In the following code, you will:
- Create a list
- Create a variable with the best value of the parameter mtry; Compulsory
- Create the loop
- Store the current value of maxnode
- Summarize the results
store_maxnode <- list() tuneGrid <- expand.grid(.mtry = best_mtry) for (maxnodes in c(5: 15)) { set.seed(1234) rf_maxnode <- train(survived~., data = data_train, method = "rf", metric = "Accuracy", tuneGrid = tuneGrid, trControl = trControl, importance = TRUE, nodesize = 14, maxnodes = maxnodes, ntree = 300) current_iteration <- toString(maxnodes) store_maxnode[[current_iteration]] <- rf_maxnode } results_mtry <- resamples(store_maxnode) summary(results_mtry)
Code explanation:
- store_maxnode <- list(): The results of the model will be stored in this list
- expand.grid(.mtry=best_mtry): Use the best value of mtry
- for (maxnodes in c(5:15)) { … }: Compute the model with values of maxnodes from 5 to 15.
- maxnodes = maxnodes: For each iteration, maxnodes equals the current loop value, i.e. 5, 6, 7, …
- current_iteration <- toString(maxnodes): Store the value of maxnodes as a string.
- store_maxnode[[current_iteration]] <- rf_maxnode: Save the result of the model in the list.
- resamples(store_maxnode): Arrange the results of the model
- summary(results_mtry): Print the summary of all the combination.
Output:
## ## Call: ## summary.resamples(object = results_mtry) ## ## Models: 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ## Number of resamples: 10 ## ## Accuracy ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 5 0.6785714 0.7529762 0.7903758 0.7799771 0.8168388 0.8433735 0 ## 6 0.6904762 0.7648810 0.7784710 0.7811962 0.8125000 0.8313253 0 ## 7 0.6904762 0.7619048 0.7738095 0.7788009 0.8102410 0.8333333 0 ## 8 0.6904762 0.7627295 0.7844234 0.7847820 0.8184524 0.8433735 0 ## 9 0.7261905 0.7747418 0.8083764 0.7955250 0.8258749 0.8333333 0 ## 10 0.6904762 0.7837780 0.7904475 0.7895869 0.8214286 0.8433735 0 ## 11 0.7023810 0.7791523 0.8024240 0.7943775 0.8184524 0.8433735 0 ## 12 0.7380952 0.7910929 0.8144005 0.8051205 0.8288511 0.8452381 0 ## 13 0.7142857 0.8005952 0.8192771 0.8075158 0.8403614 0.8452381 0 ## 14 0.7380952 0.7941050 0.8203528 0.8098967 0.8403614 0.8452381 0 ## 15 0.7142857 0.8000215 0.8203528 0.8075301 0.8378873 0.8554217 0 ## ## Kappa ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 5 0.3297872 0.4640436 0.5459706 0.5270773 0.6068751 0.6717371 0 ## 6 0.3576471 0.4981484 0.5248805 0.5366310 0.6031287 0.6480921 0 ## 7 0.3576471 0.4927448 0.5192771 0.5297159 0.5996437 0.6508314 0 ## 8 0.3576471 0.4848320 0.5408159 0.5427127 0.6200253 0.6717371 0 ## 9 0.4236277 0.5074421 0.5859472 0.5601687 0.6228626 0.6480921 0 ## 10 0.3576471 0.5255698 0.5527057 0.5497490 0.6204819 0.6717371 0 ## 11 0.3794326 0.5235007 0.5783191 0.5600467 0.6126720 0.6717371 0 ## 12 0.4460432 0.5480930 0.5999072 0.5808134 0.6296780 0.6717371 0 ## 13 0.4014252 0.5725752 0.6087279 0.5875305 0.6576219 0.6678832 0 ## 14 0.4460432 0.5585005 0.6117973 0.5911995 0.6590982 0.6717371 0 ## 15 0.4014252 0.5689401 0.6117973 0.5867010 0.6507194 0.6955990 0
The highest mean accuracy in this range (0.8099) belongs to maxnodes = 14, at the top of the tested interval. Because the best value sits at the edge of the grid, you should extend the search upwards.
store_maxnode <- list() tuneGrid <- expand.grid(.mtry = best_mtry) for (maxnodes in c(20: 30)) { set.seed(1234) rf_maxnode <- train(survived~., data = data_train, method = "rf", metric = "Accuracy", tuneGrid = tuneGrid, trControl = trControl, importance = TRUE, nodesize = 14, maxnodes = maxnodes, ntree = 300) key <- toString(maxnodes) store_maxnode[[key]] <- rf_maxnode } results_node <- resamples(store_maxnode) summary(results_node)
Output:
## ## Call: ## summary.resamples(object = results_node) ## ## Models: 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ## Number of resamples: 10 ## ## Accuracy ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 20 0.7142857 0.7821644 0.8144005 0.8075301 0.8447719 0.8571429 0 ## 21 0.7142857 0.8000215 0.8144005 0.8075014 0.8403614 0.8571429 0 ## 22 0.7023810 0.7941050 0.8263769 0.8099254 0.8328313 0.8690476 0 ## 23 0.7023810 0.7941050 0.8263769 0.8111302 0.8447719 0.8571429 0 ## 24 0.7142857 0.7946429 0.8313253 0.8135112 0.8417599 0.8690476 0 ## 25 0.7142857 0.7916667 0.8313253 0.8099398 0.8408635 0.8690476 0 ## 26 0.7142857 0.7941050 0.8203528 0.8123207 0.8528758 0.8571429 0 ## 27 0.7023810 0.8060456 0.8313253 0.8135112 0.8333333 0.8690476 0 ## 28 0.7261905 0.7941050 0.8203528 0.8111015 0.8328313 0.8690476 0 ## 29 0.7142857 0.7910929 0.8313253 0.8087063 0.8333333 0.8571429 0 ## 30 0.6785714 0.7910929 0.8263769 0.8063253 0.8403614 0.8690476 0 ## ## Kappa ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 20 0.3956835 0.5316120 0.5961830 0.5854366 0.6661120 0.6955990 0 ## 21 0.3956835 0.5699332 0.5960343 0.5853247 0.6590982 0.6919315 0 ## 22 0.3735084 0.5560661 0.6221836 0.5914492 0.6422128 0.7189781 0 ## 23 0.3735084 0.5594228 0.6228827 0.5939786 0.6657372 0.6955990 0 ## 24 0.3956835 0.5600352 0.6337821 0.5992188 0.6604703 0.7189781 0 ## 25 0.3956835 0.5530760 0.6354875 0.5912239 0.6554912 0.7189781 0 ## 26 0.3956835 0.5589331 0.6136074 0.5969142 0.6822128 0.6955990 0 ## 27 0.3735084 0.5852459 0.6368425 0.5998148 0.6426088 0.7189781 0 ## 28 0.4290780 0.5589331 0.6154905 0.5946859 0.6356141 0.7189781 0 ## 29 0.4070588 0.5534173 0.6337821 0.5901173 0.6423101 0.6919315 0 ## 30 0.3297872 0.5534173 0.6202632 0.5843432 0.6590982 0.7189781 0
The highest mean accuracy, 0.8135, is obtained with maxnodes = 24 (maxnodes = 27 ties on the mean but has a lower third quartile). You will therefore use maxnodes = 24 for the remaining steps.
Step 4) Search the best ntrees
Now that you have the best value of mtry and maxnode, you can tune the number of trees. The method is exactly the same as maxnode.
store_maxtrees <- list() for (ntree in c(250, 300, 350, 400, 450, 500, 550, 600, 800, 1000, 2000)) { set.seed(5678) rf_maxtrees <- train(survived~., data = data_train, method = "rf", metric = "Accuracy", tuneGrid = tuneGrid, trControl = trControl, importance = TRUE, nodesize = 14, maxnodes = 24, ntree = ntree) key <- toString(ntree) store_maxtrees[[key]] <- rf_maxtrees } results_tree <- resamples(store_maxtrees) summary(results_tree)
Output:
## ## Call: ## summary.resamples(object = results_tree) ## ## Models: 250, 300, 350, 400, 450, 500, 550, 600, 800, 1000, 2000 ## Number of resamples: 10 ## ## Accuracy ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 250 0.7380952 0.7976190 0.8083764 0.8087010 0.8292683 0.8674699 0 ## 300 0.7500000 0.7886905 0.8024240 0.8027199 0.8203397 0.8452381 0 ## 350 0.7500000 0.7886905 0.8024240 0.8027056 0.8277623 0.8452381 0 ## 400 0.7500000 0.7886905 0.8083764 0.8051009 0.8292683 0.8452381 0 ## 450 0.7500000 0.7886905 0.8024240 0.8039104 0.8292683 0.8452381 0 ## 500 0.7619048 0.7886905 0.8024240 0.8062914 0.8292683 0.8571429 0 ## 550 0.7619048 0.7886905 0.8083764 0.8099062 0.8323171 0.8571429 0 ## 600 0.7619048 0.7886905 0.8083764 0.8099205 0.8323171 0.8674699 0 ## 800 0.7619048 0.7976190 0.8083764 0.8110820 0.8292683 0.8674699 0 ## 1000 0.7619048 0.7976190 0.8121510 0.8086723 0.8303571 0.8452381 0 ## 2000 0.7619048 0.7886905 0.8121510 0.8086723 0.8333333 0.8452381 0 ## ## Kappa ## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ## 250 0.4061697 0.5667400 0.5836013 0.5856103 0.6335363 0.7196807 0 ## 300 0.4302326 0.5449376 0.5780349 0.5723307 0.6130767 0.6710843 0 ## 350 0.4302326 0.5449376 0.5780349 0.5723185 0.6291592 0.6710843 0 ## 400 0.4302326 0.5482030 0.5836013 0.5774782 0.6335363 0.6710843 0 ## 450 0.4302326 0.5449376 0.5780349 0.5750587 0.6335363 0.6710843 0 ## 500 0.4601542 0.5449376 0.5780349 0.5804340 0.6335363 0.6949153 0 ## 550 0.4601542 0.5482030 0.5857118 0.5884507 0.6396872 0.6949153 0 ## 600 0.4601542 0.5482030 0.5857118 0.5884374 0.6396872 0.7196807 0 ## 800 0.4601542 0.5667400 0.5836013 0.5910088 0.6335363 0.7196807 0 ## 1000 0.4601542 0.5667400 0.5961590 0.5857446 0.6343666 0.6678832 0 ## 2000 0.4601542 0.5482030 0.5961590 0.5862151 0.6440678 0.6656337 0
You have your final model. You can train the random forest with the following parameters:
- ntree = 800: 800 trees will be trained
- mtry = 4: 4 candidate features are drawn at each split
- maxnodes = 24: Each tree is limited to 24 terminal nodes (leaves)
fit_rf <- train(survived~., data_train, method = "rf", metric = "Accuracy", tuneGrid = tuneGrid, trControl = trControl, importance = TRUE, nodesize = 14, ntree = 800, maxnodes = 24)
Step 5) Evaluate the model
The library caret has a function to make prediction.
predict(model, newdata= df)
argument
- `model`: Define the model evaluated before.
- `newdata`: Define the dataset to make prediction
prediction <-predict(fit_rf, data_test)
You can use the prediction to compute the confusion matrix and see the accuracy score
confusionMatrix(prediction, data_test$survived)
Output:
## Confusion Matrix and Statistics ## ## Reference ## Prediction No Yes ## No 110 32 ## Yes 11 56 ## ## Accuracy : 0.7943 ## 95% CI : (0.733, 0.8469) ## No Information Rate : 0.5789 ## P-Value [Acc > NIR] : 3.959e-11 ## ## Kappa : 0.5638 ## Mcnemar's Test P-Value : 0.002289 ## ## Sensitivity : 0.9091 ## Specificity : 0.6364 ## Pos Pred Value : 0.7746 ## Neg Pred Value : 0.8358 ## Prevalence : 0.5789 ## Detection Rate : 0.5263 ## Detection Prevalence : 0.6794 ## Balanced Accuracy : 0.7727 ## ## 'Positive' Class : No ##
The model reaches an accuracy of 0.7943, that is 79.43 percent on the unseen test set, which is higher than the default configuration. Sensitivity is 0.9091 and specificity is 0.6364, so the model recognises non-survivors far more reliably than survivors.
Step 6) Visualize Result
Lastly, you can look at the feature importance with the varImp() function. The most important features are sex and age. That is not surprising because the important features are likely to appear closer to the root of the tree, while less important features usually appear closer to the leaves.
varImp(fit_rf)
Output:
## rf variable importance ## ## Importance ## sexmale 100.000 ## age 28.014 ## pclassMiddle 27.016 ## fare 21.557 ## pclassUpper 16.324 ## sibsp 11.246 ## parch 5.522 ## embarkedC 4.908 ## embarkedQ 1.420 ## embarkedS 0.000
Random Forest in R: Quick Function Reference
The table below lists every function used across the six steps, the package that supplies it, and the parameters it expects.
| Library | Objective | Function | Parameter |
|---|---|---|---|
| randomForest | Create a random forest | randomForest() | formula, ntree=n, mtry=FALSE, maxnodes = NULL |
| caret | Create k-fold cross validation | trainControl() | method = “cv”, number = n, search =”grid” |
| caret | Train a Random Forest | train() | formula, df, method = “rf”, metric= “Accuracy”, trControl = trainControl(), tuneGrid = NULL |
| caret | Predict out of sample | predict | model, newdata= df |
| caret | Confusion Matrix and Statistics | confusionMatrix() | model, y test |
| caret | Variable importance | varImp() | model |
Appendix: Models Available in caret
The train() function can fit far more than random forests. Run the command below to print every model identifier that caret supports, then pass any of them to the method argument.
names(getModelInfo())
Output:
## [1] "ada" "AdaBag" "AdaBoost.M1" ## [4] "adaboost" "amdai" "ANFIS" ## [7] "avNNet" "awnb" "awtan" ## [10] "bag" "bagEarth" "bagEarthGCV" ## [13] "bagFDA" "bagFDAGCV" "bam" ## [16] "bartMachine" "bayesglm" "binda" ## [19] "blackboost" "blasso" "blassoAveraged" ## [22] "bridge" "brnn" "BstLm" ## [25] "bstSm" "bstTree" "C5.0" ## [28] "C5.0Cost" "C5.0Rules" "C5.0Tree" ## [31] "cforest" "chaid" "CSimca" ## [34] "ctree" "ctree2" "cubist" ## [37] "dda" "deepboost" "DENFIS" ## [40] "dnn" "dwdLinear" "dwdPoly" ## [43] "dwdRadial" "earth" "elm" ## [46] "enet" "evtree" "extraTrees" ## [49] "fda" "FH.GBML" "FIR.DM" ## [52] "foba" "FRBCS.CHI" "FRBCS.W" ## [55] "FS.HGD" "gam" "gamboost" ## [58] "gamLoess" "gamSpline" "gaussprLinear" ## [61] "gaussprPoly" "gaussprRadial" "gbm_h3o" ## [64] "gbm" "gcvEarth" "GFS.FR.MOGUL" ## [67] "GFS.GCCL" "GFS.LT.RS" "GFS.THRIFT" ## [70] "glm.nb" "glm" "glmboost" ## [73] "glmnet_h3o" "glmnet" "glmStepAIC" ## [76] "gpls" "hda" "hdda" ## [79] "hdrda" "HYFIS" "icr" ## [82] "J48" "JRip" "kernelpls" ## [85] "kknn" "knn" "krlsPoly" ## [88] "krlsRadial" "lars" "lars2" ## [91] "lasso" "lda" "lda2" ## [94] "leapBackward" "leapForward" "leapSeq" ## [97] "Linda" "lm" "lmStepAIC" ## [100] "LMT" "loclda" "logicBag" ## [103] "LogitBoost" "logreg" "lssvmLinear" ## [106] "lssvmPoly" "lssvmRadial" "lvq" ## [109] "M5" "M5Rules" "manb" ## [112] "mda" "Mlda" "mlp" ## [115] "mlpKerasDecay" "mlpKerasDecayCost" "mlpKerasDropout" ## [118] "mlpKerasDropoutCost" "mlpML" "mlpSGD" ## [121] "mlpWeightDecay" "mlpWeightDecayML" "monmlp" ## [124] "msaenet" "multinom" "mxnet" ## [127] "mxnetAdam" "naive_bayes" "nb" ## [130] "nbDiscrete" "nbSearch" "neuralnet" ## [133] "nnet" "nnls" "nodeHarvest" ## [136] "null" "OneR" "ordinalNet" ## [139] "ORFlog" "ORFpls" "ORFridge" ## [142] "ORFsvm" "ownn" "pam" ## [145] "parRF" "PART" "partDSA" ## [148] "pcaNNet" "pcr" "pda" ## [151] "pda2" "penalized" "PenalizedLDA" ## [154] "plr" "pls" "plsRglm" ## [157] "polr" "ppr" "PRIM" ## [160] "protoclass" "pythonKnnReg" "qda" ## [163] "QdaCov" "qrf" "qrnn" ## [166] "randomGLM" "ranger" "rbf" ## [169] "rbfDDA" "Rborist" "rda" ## [172] "regLogistic" "relaxo" "rf" ## [175] "rFerns" "RFlda" "rfRules" ## [178] "ridge" "rlda" "rlm" ## [181] "rmda" "rocc" "rotationForest" ## [184] "rotationForestCp" "rpart" "rpart1SE" ## [187] "rpart2" "rpartCost" "rpartScore" ## [190] "rqlasso" "rqnc" "RRF" ## [193] "RRFglobal" "rrlda" "RSimca" ## [196] "rvmLinear" "rvmPoly" "rvmRadial" ## [199] "SBC" "sda" "sdwd" ## [202] "simpls" "SLAVE" "slda" ## [205] "smda" "snn" "sparseLDA" ## [208] "spikeslab" "spls" "stepLDA" ## [211] "stepQDA" "superpc" "svmBoundrangeString"## [214] "svmExpoString" "svmLinear" "svmLinear2" ## [217] "svmLinear3" "svmLinearWeights" "svmLinearWeights2" ## [220] "svmPoly" "svmRadial" "svmRadialCost" ## [223] "svmRadialSigma" "svmRadialWeights" "svmSpectrumString" ## [226] "tan" "tanSearch" "treebag" ## [229] "vbmpRadial" "vglmAdjCat" "vglmContRatio" ## [232] "vglmCumulative" "widekernelpls" "WM" ## [235] "wsrf" "xgbLinear" "xgbTree" ## [238] "xyf"
