  To obtain good performance, convolutional neural networks are usually
over-parameterized. This phenomenon has stimulated two interesting topics:
pruning the unimportant weights for compression and reactivating the
unimportant weights to make full use of network capability. However, current
weight reactivation methods usually reactivate the entire filters, which may
not be precise enough. Looking back in history, the prosperity of filter
pruning is mainly due to its friendliness to hardware implementation, but
pruning at a finer structure level, i.e., weight elements, usually leads to
better network performance. We study the problem of weight element reactivation
in this paper. Motivated by evolution, we select the unimportant filters and
update their unimportant elements by combining them with the important elements
of important filters, just like gene crossover to produce better offspring, and
the proposed method is called weight evolution (WE). WE is mainly composed of
four strategies. We propose a global selection strategy and a local selection
strategy and combine them to locate the unimportant filters. A forward matching
strategy is proposed to find the matched important filters and a crossover
strategy is proposed to utilize the important elements of the important filters
for updating unimportant filters. WE is plug-in to existing network
architectures. Comprehensive experiments show that WE outperforms the other
reactivation methods and plug-in training methods with typical convolutional
neural networks, especially lightweight networks. Our code is available at
https://github.com/BZQLin/Weight-evolution.
