  In this paper, we present a novel deep learning approach, deeply-fused nets.
The central idea of our approach is deep fusion, i.e., combine the intermediate
representations of base networks, where the fused output serves as the input of
the remaining part of each base network, and perform such combinations deeply
over several intermediate representations. The resulting deeply fused net
enjoys several benefits. First, it is able to learn multi-scale representations
as it enjoys the benefits of more base networks, which could form the same
fused network, other than the initial group of base networks. Second, in our
suggested fused net formed by one deep and one shallow base networks, the flows
of the information from the earlier intermediate layer of the deep base network
to the output and from the input to the later intermediate layer of the deep
base network are both improved. Last, the deep and shallow base networks are
jointly learnt and can benefit from each other. More interestingly, the
essential depth of a fused net composed from a deep base network and a shallow
base network is reduced because the fused net could be composed from a less
deep base network, and thus training the fused net is less difficult than
training the initial deep base network. Empirical results demonstrate that our
approach achieves superior performance over two closely-related methods, ResNet
and Highway, and competitive performance compared to the state-of-the-arts.
