  An information theoretic approach to learning the complexity of
classification and regression trees and the number of trees in gradient tree
boosting is proposed. The optimism (test loss minus training loss) of the
greedy leaf splitting procedure is shown to be the maximum of a
Cox-Ingersoll-Ross process, from which a generalization-error based information
criterion is formed. The proposed procedure allows fast local model selection
without cross validation based hyper parameter tuning, and hence efficient and
automatic comparison among the large number of models performed during each
boosting iteration. Relative to xgboost, speedups on numerical experiments
ranges from around 10 to about 1400, at similar predictive-power measured in
terms of test-loss.
