  We present a new approach and a novel architecture, termed WSNet, for
learning compact and efficient deep neural networks. Existing approaches
conventionally learn full model parameters independently and then compress them
via ad hoc processing such as model pruning or filter factorization.
Alternatively, WSNet proposes learning model parameters by sampling from a
compact set of learnable parameters, which naturally enforces {parameter
sharing} throughout the learning process. We demonstrate that such a novel
weight sampling approach (and induced WSNet) promotes both weights and
computation sharing favorably. By employing this method, we can more
efficiently learn much smaller networks with competitive performance compared
to baseline networks with equal numbers of convolution filters. Specifically,
we consider learning compact and efficient 1D convolutional neural networks for
audio classification. Extensive experiments on multiple audio classification
datasets verify the effectiveness of WSNet. Combined with weight quantization,
the resulted models are up to 180 times smaller and theoretically up to 16
times faster than the well-established baselines, without noticeable
performance drop.
