  Understanding why and how certain neural networks outperform others is key to
guiding future development of network architectures and optimization methods.
To this end, we introduce a novel visualization algorithm that reveals the
internal geometry of such networks: Multislice PHATE (M-PHATE), the first
method designed explicitly to visualize how a neural network's hidden
representations of data evolve throughout the course of training. We
demonstrate that our visualization provides intuitive, detailed summaries of
the learning dynamics beyond simple global measures (i.e., validation loss and
accuracy), without the need to access validation data. Furthermore, M-PHATE
better captures both the dynamics and community structure of the hidden units
as compared to visualization based on standard dimensionality reduction methods
(e.g., ISOMAP, t-SNE). We demonstrate M-PHATE with two vignettes: continual
learning and generalization. In the former, the M-PHATE visualizations display
the mechanism of "catastrophic forgetting" which is a major challenge for
learning in task-switching contexts. In the latter, our visualizations reveal
how increased heterogeneity among hidden units correlates with improved
generalization performance. An implementation of M-PHATE, along with scripts to
reproduce the figures in this paper, is available at
https://github.com/scottgigante/M-PHATE.
