  In this article, we take one step toward understanding the learning behavior
of deep residual networks, and supporting the observation that deep residual
networks behave like ensembles. We propose a new convolutional neural network
architecture which builds upon the success of residual networks by explicitly
exploiting the interpretation of very deep networks as an ensemble. The
proposed multi-residual network increases the number of residual functions in
the residual blocks. Our architecture generates models that are wider, rather
than deeper, which significantly improves accuracy. We show that our model
achieves an error rate of 3.73% and 19.45% on CIFAR-10 and CIFAR-100
respectively, that outperforms almost all of the existing models. We also
demonstrate that our model outperforms very deep residual networks by 0.22%
(top-1 error) on the full ImageNet 2012 classification dataset. Additionally,
inspired by the parallel structure of multi-residual networks, a model
parallelism technique has been investigated. The model parallelism method
distributes the computation of residual blocks among the processors, yielding
up to 15% computational complexity improvement.
