  Convolutional Neural networks nowadays are of tremendous importance for any
image classification system. One of the most investigated methods to increase
the accuracy of CNN is by increasing the depth of CNN. Increasing the depth by
stacking more layers also increases the difficulty of training besides making
it computationally expensive. Some research found that adding auxiliary forks
after intermediate layers increases the accuracy. Specifying which intermediate
layer shoud have the fork just addressed recently. Where a simple rule were
used to detect the position of intermediate layers that needs the auxiliary
supervision fork. This technique known as convolutional neural networks with
deep supervision (CNDS). This technique enhanced the accuracy of classification
over the straight forward CNN used on the MIT places dataset and ImageNet. In
the other side, Residual Learning is another technique emerged recently to ease
the training of very deep CNN. Residual Learning framwork changed the learning
of layers from unreferenced functions to learning residual function with regard
to the layer's input. Residual Learning achieved state of arts results on
ImageNet 2015 and COCO competitions. In this paper, we study the effect of
adding residual connections to CNDS network. Our experiments results show
increasing of accuracy over using CNDS only.
