  Network in Netwrok (NiN) is an effective instance and an important extension
of Convolutional Neural Network (CNN) consisting of alternating convolutional
layers and pooling layers. Instead of using a linear filter for convolution,
NiN utilizes shallow MultiLayer Perceptron (MLP), a nonlinear function, to
replace the linear filter. Because of the powerfulness of MLP and $ 1\times 1 $
convolutions in spatial domain, NiN has stronger ability of feature
representation and hence results in better recognition rate. However, MLP
itself consists of fully connected layers which give rise to a large number of
parameters. In this paper, we propose to replace dense shallow MLP with sparse
shallow MLP. One or more layers of the sparse shallow MLP are sparely connected
in the channel dimension or channel-spatial domain. The proposed method is
implemented by applying unshared convolution across the channel dimension and
applying shared convolution across the spatial dimension in some computational
layers. The proposed method is called CiC. Experimental results on the CIFAR10
dataset, augmented CIFAR10 dataset, and CIFAR100 dataset demonstrate the
effectiveness of the proposed CiC method.
