  A deep residual network, built by stacking a sequence of residual blocks, is
easy to train, because identity mappings skip residual branches and thus
improve information flow. To further reduce the training difficulty, we present
a simple network architecture, deep merge-and-run neural networks. The novelty
lies in a modularized building block, merge-and-run block, which assembles
residual branches in parallel through a merge-and-run mapping: Average the
inputs of these residual branches (Merge), and add the average to the output of
each residual branch as the input of the subsequent residual branch (Run),
respectively. We show that the merge-and-run mapping is a linear idempotent
function in which the transformation matrix is idempotent, and thus improves
information flow, making training easy. In comparison to residual networks, our
networks enjoy compelling advantages: they contain much shorter paths, and the
width, i.e., the number of channels, is increased. We evaluate the performance
on the standard recognition tasks. Our approach demonstrates consistent
improvements over ResNets with the comparable setup, and achieves competitive
results (e.g., $3.57\%$ testing error on CIFAR-$10$, $19.00\%$ on CIFAR-$100$,
$1.51\%$ on SVHN).
