  A new extremely simple ensemble-based model with the uniformly generated
axis-parallel hyper-rectangles as base models (HRBM) is proposed. Two types of
HRBMs are studied: closed rectangles and corners. The main idea behind HRBM is
to consider and count training examples inside and outside each rectangle. It
is proposed to incorporate HRBMs into the gradient boosting machine (GBM).
Despite simplicity of HRBMs, it turns out that these simple base models allow
us to construct effective ensemble-based models and avoid overfitting. A simple
method for calculating optimal regularization parameters of the ensemble-based
model, which can be modified in the explicit way at each iteration of GBM, is
considered. Moreover, a new regularization called the "step height penalty" is
studied in addition to the standard L1 and L2 regularizations. An extremely
simple approach to the proposed ensemble-based model prediction interpretation
by using the well-known method SHAP is proposed. It is shown that GBM with HRBM
can be regarded as a model extending a set of interpretable models for
explaining black-box models. Numerical experiments with real datasets
illustrate the proposed GBM with HRBMs for regression and classification
problems. Experiments also illustrate computational efficiency of the proposed
SHAP modifications. The code of proposed algorithms implementing GBM with HRBM
is publicly available.
