  Classification trees continue to be widely adopted in machine learning
applications due to their inherently interpretable nature and scalability. We
propose a rolling subtree lookahead algorithm that combines the relative
scalability of the myopic approaches with the foresight of the optimal
approaches in constructing trees. The limited foresight embedded in our
algorithm mitigates the learning pathology observed in optimal approaches. At
the heart of our algorithm lies a novel two-depth optimal binary classification
tree formulation flexible to handle any loss function. We show that the
feasible region of this formulation is an integral polyhedron, yielding the LP
relaxation solution optimal. Through extensive computational analyses, we
demonstrate that our approach outperforms optimal and myopic approaches in 808
out of 1330 problem instances, improving the out-of-sample accuracy by up to
23.6% and 14.4%, respectively.
