  Recent studies have demonstrated that the convolutional networks heavily rely
on the quality and quantity of generated features. However, in lightweight
networks, there are limited available feature information because these
networks tend to be shallower and thinner due to the efficiency consideration.
For farther improving the performance and accuracy of lightweight networks, we
develop Super Interaction Neural Networks (SINet) model from a novel point of
view: enhancing the information interaction in neural networks. In order to
achieve information interaction along the width of the deep network, we propose
Exchange Shortcut Connection, which can integrate the information from
different convolution groups without any extra computation cost. And then, in
order to achieve information interaction along the depth of the network, we
proposed Dense Funnel Layer and Attention based Hierarchical Joint Decision,
which are able to make full use of middle layer features. Our experiments show
that the superior performance of SINet over other state-of-the-art lightweight
models in ImageNet dataset. Furthermore, we also exhibit the effectiveness and
universality of our proposed components by ablation studies.
