  The interpretability of models has become a crucial issue in Machine Learning
because of algorithmic decisions' growing impact on real-world applications.
Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning
tools for classification tasks. However, while combining multiple trees may
provide higher prediction quality than a single one, it sacrifices the
interpretability property resulting in "black-box" models. In light of this, we
aim to develop an interpretable representation of a tree-ensemble model that
can provide valuable insights into its behavior. First, given a target
tree-ensemble model, we develop a hierarchical visualization tool based on a
heatmap representation of the forest's feature use, considering the frequency
of a feature and the level at which it is selected as an indicator of
importance. Next, we propose a mixed-integer linear programming (MILP)
formulation for constructing a single optimal multivariate tree that accurately
mimics the target model predictions. The goal is to provide an interpretable
surrogate model based on oblique hyperplane splits, which uses only the most
relevant features according to the defined forest's importance indicators. The
MILP model includes a penalty on feature selection based on their frequency in
the forest to further induce sparsity of the splits. The natural formulation
has been strengthened to improve the computational performance of
{mixed-integer} software. Computational experience is carried out on benchmark
datasets from the UCI repository using a state-of-the-art off-the-shelf solver.
Results show that the proposed model is effective in yielding a shallow
interpretable tree approximating the tree-ensemble decision function.
