  Safe use of Deep Neural Networks (DNNs) requires careful testing. However,
deployed models are often trained further to improve in performance. As
rigorous testing and evaluation is expensive, triggers are in need to determine
the degree of change of a model. In this paper we investigate the weight space
of DNN models for structure that can be exploited to that end. Our results show
that DNN models evolve on unique, smooth trajectories in weight space which can
be used to track DNN training progress. We hypothesize that curvature and
smoothness of the trajectories as well as step length along it may contain
information on the state of training as well as potential domain shifts. We
show that the model trajectories can be separated and the order of checkpoints
on the trajectories recovered, which may serve as a first step towards DNN
model versioning.
