  In recent years, deep neural networks have been applied to obtain high
performance of prediction, classification, and pattern recognition. However,
the weights in these deep neural networks are difficult to be explained.
Although a linear regression method can provide explainable results, the method
is not suitable in the case of input interaction. Therefore, an explainable
deep neural network (ExpDNN) with explainable layers is proposed to obtain
explainable results in the case of input interaction. Three cases were given to
evaluate the proposed ExpDNN, and the results showed that the absolute value of
weight in an explainable layer can be used to explain the weight of
corresponding input for feature extraction.
