  Artificial Neural Networks form the basis of very powerful learning methods.
It has been observed that a naive application of fully connected neural
networks to data with many irrelevant variables often leads to overfitting. In
an attempt to circumvent this issue, a prior knowledge pertaining to what
features are relevant and their possible feature interactions can be encoded
into these networks. In this work, we use decision trees to capture such
relevant features and their interactions and define a mapping to encode
extracted relationships into a neural network. This addresses the
initialization related concern of fully connected neural networks. At the same
time through feature selection it enables learning of compact representations
compared to state of the art tree-based approaches. Empirical evaluations and
simulation studies show the superiority of such an approach over fully
connected neural networks and tree-based approaches
