  Deep residual networks have recently shown appealing performance on many
challenging computer vision tasks. However, the original residual structure
still has some defects making it difficult to converge on very deep networks.
In this paper, we introduce a weighted residual network to address the
incompatibility between \texttt{ReLU} and element-wise addition and the deep
network initialization problem. The weighted residual network is able to learn
to combine residuals from different layers effectively and efficiently. The
proposed models enjoy a consistent improvement over accuracy and convergence
with increasing depths from 100+ layers to 1000+ layers. Besides, the weighted
residual networks have little more computation and GPU memory burden than the
original residual networks. The networks are optimized by projected stochastic
gradient descent. Experiments on CIFAR-10 have shown that our algorithm has a
\emph{faster convergence speed} than the original residual networks and reaches
a \emph{high accuracy} at 95.3\% with a 1192-layer model.
