  A core challenge in the interpretation of deep neural networks is identifying
commonalities between the underlying algorithms implemented by distinct
networks trained for the same task. Motivated by this problem, we introduce
DYNAMO, an algorithm that constructs low-dimensional manifolds where each point
corresponds to a neural network model, and two points are nearby if the
corresponding neural networks enact similar high-level computational processes.
DYNAMO takes as input a collection of pre-trained neural networks and outputs a
meta-model that emulates the dynamics of the hidden states as well as the
outputs of any model in the collection. The specific model to be emulated is
determined by a model embedding vector that the meta-model takes as input;
these model embedding vectors constitute a manifold corresponding to the given
population of models. We apply DYNAMO to both RNNs and CNNs, and find that the
resulting model embedding spaces enable novel applications: clustering of
neural networks on the basis of their high-level computational processes in a
manner that is less sensitive to reparameterization; model averaging of several
neural networks trained on the same task to arrive at a new, operable neural
network with similar task performance; and semi-supervised learning via
optimization on the model embedding space. Using a fixed-point analysis of
meta-models trained on populations of RNNs, we gain new insights into how
similarities of the topology of RNN dynamics correspond to similarities of
their high-level computational processes.
