  Very large-scale Deep Neural Networks (DNNs) have achieved remarkable
successes in a large variety of computer vision tasks. However, the high
computation intensity of DNNs makes it challenging to deploy these models on
resource-limited systems. Some studies used low-rank approaches that
approximate the filters by low-rank basis to accelerate the testing. Those
works directly decomposed the pre-trained DNNs by Low-Rank Approximations
(LRA). How to train DNNs toward lower-rank space for more efficient DNNs,
however, remains as an open area. To solve the issue, in this work, we propose
Force Regularization, which uses attractive forces to enforce filters so as to
coordinate more weight information into lower-rank space. We mathematically and
empirically verify that after applying our technique, standard LRA methods can
reconstruct filters using much lower basis and thus result in faster DNNs. The
effectiveness of our approach is comprehensively evaluated in ResNets, AlexNet,
and GoogLeNet. In AlexNet, for example, Force Regularization gains 2x speedup
on modern GPU without accuracy loss and 4.05x speedup on CPU by paying small
accuracy degradation. Moreover, Force Regularization better initializes the
low-rank DNNs such that the fine-tuning can converge faster toward higher
accuracy. The obtained lower-rank DNNs can be further sparsified, proving that
Force Regularization can be integrated with state-of-the-art sparsity-based
acceleration methods. Source code is available in
https://github.com/wenwei202/caffe
