  We propose a novel tree-based ensemble method named Selective Cascade of
Residual ExtraTrees (SCORE). SCORE draws inspiration from representation
learning, incorporates regularized regression with variable selection features,
and utilizes boosting to improve prediction and reduce generalization errors.
We also develop a variable importance measure to increase the explainability of
SCORE. Our computer experiments show that SCORE provides comparable or superior
performance in prediction against ExtraTrees, random forest, gradient boosting
machine, and neural networks; and the proposed variable importance measure for
SCORE is comparable to studied benchmark methods. Finally, the predictive
performance of SCORE remains stable across hyper-parameter values, suggesting
potential robustness to hyperparameter specification.
