  Graph neural networks (GNNs) are powerful models that have been successful in
various graph representation learning tasks. Whereas gradient boosted decision
trees (GBDT) often outperform other machine learning methods when faced with
heterogeneous tabular data. But what approach should be used for graphs with
tabular node features? Previous GNN models have mostly focused on networks with
homogeneous sparse features and, as we show, are suboptimal in the
heterogeneous setting. In this work, we propose a novel architecture that
trains GBDT and GNN jointly to get the best of both worlds: the GBDT model
deals with heterogeneous features, while GNN accounts for the graph structure.
Our model benefits from end-to-end optimization by allowing new trees to fit
the gradient updates of GNN. With an extensive experimental comparison to the
leading GBDT and GNN models, we demonstrate a significant increase in
performance on a variety of graphs with tabular features. The code is
available: https://github.com/nd7141/bgnn.
