  Deep neural networks have been proven powerful at processing perceptual data,
such as images and audio. However for tabular data, tree-based models are more
popular. A nice property of tree-based models is their natural
interpretability. In this work, we present Deep Neural Decision Trees (DNDT) --
tree models realised by neural networks. A DNDT is intrinsically interpretable,
as it is a tree. Yet as it is also a neural network (NN), it can be easily
implemented in NN toolkits, and trained with gradient descent rather than
greedy splitting. We evaluate DNDT on several tabular datasets, verify its
efficacy, and investigate similarities and differences between DNDT and vanilla
decision trees. Interestingly, DNDT self-prunes at both split and
feature-level.
