  In this paper, we study the problem of designing efficient convolutional
neural network architectures with the interest in eliminating the redundancy in
convolution kernels. In addition to structured sparse kernels, low-rank kernels
and the product of low-rank kernels, the product of structured sparse kernels,
which is a framework for interpreting the recently-developed interleaved group
convolutions (IGC) and its variants (e.g., Xception), has been attracting
increasing interests.
  Motivated by the observation that the convolutions contained in a group
convolution in IGC can be further decomposed in the same manner, we present a
modularized building block, {IGCV$2$:} interleaved structured sparse
convolutions. It generalizes interleaved group convolutions, which is composed
of two structured sparse kernels, to the product of more structured sparse
kernels, further eliminating the redundancy. We present the complementary
condition and the balance condition to guide the design of structured sparse
kernels, obtaining a balance among three aspects: model size, computation
complexity and classification accuracy. Experimental results demonstrate the
advantage on the balance among these three aspects compared to interleaved
group convolutions and Xception, and competitive performance compared to other
state-of-the-art architecture design methods.
