  We introduce WildWood (WW), a new ensemble algorithm for supervised learning
of Random Forest (RF) type. While standard RF algorithms use bootstrap
out-of-bag samples to compute out-of-bag scores, WW uses these samples to
produce improved predictions given by an aggregation of the predictions of all
possible subtrees of each fully grown tree in the forest. This is achieved by
aggregation with exponential weights computed over out-of-bag samples, that are
computed exactly and very efficiently thanks to an algorithm called context
tree weighting. This improvement, combined with a histogram strategy to
accelerate split finding, makes WW fast and competitive compared with other
well-established ensemble methods, such as standard RF and extreme gradient
boosting algorithms.
