  The most difficult task in machine learning is to interpret trained shallow
neural networks. Deep neural networks (DNNs) provide impressive results on a
larger number of tasks, but it is generally still unclear how decisions are
made by such a trained deep neural network. Providing feature importance is the
most important and popular interpretation technique used in shallow and deep
neural networks. In this paper, we develop an algorithm extending the idea of
Garson Algorithm to explain Deep Belief Network based Auto-encoder (DBNA). It
is used to determine the contribution of each input feature in the DBN. It can
be used for any kind of neural network with many hidden layers. The
effectiveness of this method is tested on both classification and regression
datasets taken from literature. Important features identified by this method
are compared against those obtained by Wald chi square (\c{hi}2). For 2 out of
4 classification datasets and 2 out of 5 regression datasets, our proposed
methodology resulted in the identification of better-quality features leading
to statistically more significant results vis-\`a-vis Wald \c{hi}2.
