  This paper introduces a concept of layer aggregation to describe how
information from previous layers can be reused to better extract features at
the current layer. While DenseNet is a typical example of the layer aggregation
mechanism, its redundancy has been commonly criticized in the literature. This
motivates us to propose a very light-weighted module, called recurrent layer
aggregation (RLA), by making use of the sequential structure of layers in a
deep CNN. Our RLA module is compatible with many mainstream deep CNNs,
including ResNets, Xception and MobileNetV2, and its effectiveness is verified
by our extensive experiments on image classification, object detection and
instance segmentation tasks. Specifically, improvements can be uniformly
observed on CIFAR, ImageNet and MS COCO datasets, and the corresponding
RLA-Nets can surprisingly boost the performances by 2-3% on the object
detection task. This evidences the power of our RLA module in helping main CNNs
better learn structural information in images.
