  Decades after their inception, random forests continue to provide
state-of-the-art accuracy in a variety of learning problems, outperforming in
this respect alternative machine learning algorithms such as decision trees or
even neural networks. However, being an ensemble method, the one aspect where
random forests tend to severely underperform decision trees is
interpretability. In the present work, we propose a post-hoc approach that aims
to have the best of both worlds: the accuracy of random forests and the
interpretability of decision trees. To this end, we present two forest-pruning
methods to find an optimal sub-forest within a given random forest, and then,
when applicable, combine the selected trees into one. Our first method relies
on constrained exhaustive search, while our second method is based on an
adaptation of the LASSO methodology. Extensive experiments over synthetic and
real world datasets show that, in the majority of scenarios, at least one of
the two methods proposed is more accurate than the original random forest,
while just using a small fraction of the trees, aiding result interpretability.
Compared to current state-of-the-art forest pruning methods, namely sequential
forward selection and (a variation of) sequential backward selection, our
methods tend to outperform both of them, whether in terms of accuracy, number
of trees employed, or both.
