  In this paper we propose a method to build a neural network that is similar
to an ensemble of decision trees. We first illustrate how to convert a learned
ensemble of decision trees to a single neural network with one hidden layer and
an input transformation. We then relax some properties of this network such as
thresholds and activation functions to train an approximately equivalent
decision tree ensemble. The final model, Hammock, is surprisingly simple: a
fully connected two layers neural network where the input is quantized and
one-hot encoded. Experiments on large and small datasets show this simple
method can achieve performance similar to that of Gradient Boosted Decision
Trees.
