  Deep neural networks are able to learn multi-layered representation via back
propagation (BP). Although the gradient boosting decision tree (GBDT) is
effective for modeling tabular data, it is non-differentiable with respect to
its input, thus suffering from learning multi-layered representation. In this
paper, we propose a framework of learning multi-layered GBDT via BP. We
approximate the gradient of GBDT based on linear regression. Specifically, we
use linear regression to replace the constant value at each leaf ignoring the
contribution of individual samples to the tree structure. In this way, we
estimate the gradient for intermediate representations, which facilitates BP
for multi-layered GBDT. Experiments show the effectiveness of the proposed
method in terms of performance and representation ability. To the best of our
knowledge, this is the first work of optimizing multi-layered GBDT via BP. This
work provides a new possibility of exploring deep tree based learning and
combining GBDT with neural networks.
