  Deep learning as represented by the artificial deep neural networks (DNNs)
has achieved great success in many important areas that deal with text, images,
videos, graphs, and so on. However, the black-box nature of DNNs has become one
of the primary obstacles for their wide acceptance in mission-critical
applications such as medical diagnosis and therapy. Due to the huge potential
of deep learning, interpreting neural networks has recently attracted much
research attention. In this paper, based on our comprehensive taxonomy, we
systematically review recent studies in understanding the mechanism of neural
networks, describe applications of interpretability especially in medicine, and
discuss future directions of interpretability research, such as in relation to
fuzzy logic and brain science.
