  Convolutional neural networks (CNNs) are inherently suffering from massively
redundant computation (FLOPs) due to the dense connection pattern between
feature maps and convolution kernels. Recent research has investigated the
sparse relationship between channels, however, they ignored the spatial
relationship within a channel. In this paper, we present a novel convolutional
operator, namely comb convolution, to exploit the intra-channel sparse
relationship among neurons. The proposed convolutional operator eliminates
nearly 50% of connections by inserting uniform mappings into standard
convolutions and removing about half of spatial connections in convolutional
layer. Notably, our work is orthogonal and complementary to existing methods
that reduce channel-wise redundancy. Thus, it has great potential to further
increase efficiency through integrating the comb convolution to existing
architectures. Experimental results demonstrate that by simply replacing
standard convolutions with comb convolutions on state-of-the-art CNN
architectures (e.g., VGGNets, Xception and SE-Net), we can achieve 50% FLOPs
reduction while still maintaining the accuracy.
