  Deep neural networks (DNNs), the agents of deep learning (DL), require a
massive number of parallel/sequential operations. This makes it difficult to
comprehend DNNs' operations and impedes proper diagnosis. Without better
knowledge of their internal process, deploying DNNs in high-stakes domains can
lead to catastrophic failures. Therefore, to build more reliable DNNs/DL to be
deployed in high-stakes real-world problems, it is imperative that we gain
insights into DNNs' internal operations underlying their decision-making. Here,
we use the self-organizing map (SOM) to analyze DL models' internal codes
associated with DNNs' decision-making. Our analyses suggest that shallow layers
close to the input layer compress features into condensed space and that deep
layers close to the output layer expand feature space. We also found evidence
indicating that compressed features may underlie DNNs' vulnerabilities to
adversarial perturbations.
