  This paper shows that decision trees constructed with Classification and
Regression Trees (CART) and C4.5 methodology are consistent for regression and
classification tasks, even when the number of predictor variables grows
sub-exponentially with the sample size, under natural 0-norm and 1-norm
sparsity constraints. The theory applies to a wide range of models, including
(ordinary or logistic) additive regression models with component functions that
are continuous, of bounded variation, or, more generally, Borel measurable.
Consistency holds for arbitrary joint distributions of the predictor variables,
thereby accommodating continuous, discrete, and/or dependent data. Finally, we
show that these qualitative properties of individual trees are inherited by
Breiman's random forests. A key step in the analysis is the establishment of an
oracle inequality, which allows for a precise characterization of the
goodness-of-fit and complexity tradeoff for a mis-specified model.
