  Convolutional Neural Networks (CNNs) have achieved remarkable success in
various computer vision tasks but rely on tremendous computational cost. To
solve this problem, existing approaches either compress well-trained
large-scale models or learn lightweight models with carefully designed network
structures. In this work, we make a close study of the convolution operator,
which is the basic unit used in CNNs, to reduce its computing load. In
particular, we propose a compact convolution module, called CompConv, to
facilitate efficient feature learning. With the divide-and-conquer strategy,
CompConv is able to save a great many computations as well as parameters to
produce a certain dimensional feature map. Furthermore, CompConv discreetly
integrates the input features into the outputs to efficiently inherit the input
information. More importantly, the novel CompConv is a plug-and-play module
that can be directly applied to modern CNN structures to replace the vanilla
convolution layers without further effort. Extensive experimental results
suggest that CompConv can adequately compress the benchmark CNN structures yet
barely sacrifice the performance, surpassing other competitors.
