Skip to main content

What is the difference between a random forest and gradient boosting?

Machine learning basics · Intermediate 2

Short answer

Both combine many decision trees. A random forest trains deep trees independently on random samples of rows and features and averages them, which mainly cuts variance. Gradient boosting trains shallow trees one after another, each fixing the errors of the ones before, which mainly cuts bias. Boosting is often more accurate on tables but needs more careful tuning.

Deeper answer

Random forests are forgiving: sensible defaults work, they rarely overfit badly as you add trees, and training parallelises. Gradient boosting libraries such as XGBoost and LightGBM usually win on tabular data competitions, but you must tune the learning rate, number of trees, tree depth and regularisation, and use early stopping on a validation set because adding trees can overfit.

Both give feature importance scores, though these can mislead with correlated features; permutation importance or SHAP values are more reliable. For a first model on a table, a random forest is a strong baseline; gradient boosting is usually the one you ship after tuning.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

Trees and gradient boosting

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.