  We train deep residual networks with a stochastic variant of the nonlinear
multigrid method MG/OPT. To build the multilevel hierarchy, we use the
dynamical systems viewpoint specific to residual networks. We report
significant speed-ups and additional robustness for training MNIST on deep
residual networks. Our numerical experiments also indicate that multilevel
training can be used as a pruning technique, as many of the auxiliary networks
have accuracies comparable to the original network.
