  Dynamic convolution achieves better performance for efficient CNNs at the
cost of negligible FLOPs increase. However, the performance increase can not
match the significantly expanded number of parameters, which is the main
bottleneck in real-world applications. Contrastively, mask-based unstructured
pruning obtains a lightweight network by removing redundancy in the heavy
network. In this paper, we propose a new framework, \textbf{Sparse Dynamic
Convolution} (\textsc{SD-Conv}), to naturally integrate these two paths such
that it can inherit the advantage of dynamic mechanism and sparsity. We first
design a binary mask derived from a learnable threshold to prune static
kernels, significantly reducing the parameters and computational cost but
achieving higher performance in Imagenet-1K. We further transfer pretrained
models into a variety of downstream tasks, showing consistently better results
than baselines. We hope our SD-Conv could be an efficient alternative to
conventional dynamic convolutions.
