  Conventional NAS-based pruning algorithms aim to find the sub-network with
the best validation performance. However, validation performance does not
successfully represent test performance, i.e., potential performance. Also,
although fine-tuning the pruned network to restore the performance drop is an
inevitable process, few studies have handled this issue. This paper provides a
novel Ensemble Knowledge Guidance (EKG) to solve both problems at once. First,
we experimentally prove that the fluctuation of loss landscape can be an
effective metric to evaluate the potential performance. In order to search a
sub-network with the smoothest loss landscape at a low cost, we employ EKG as a
search reward. EKG utilized for the following search iteration is composed of
the ensemble knowledge of interim sub-networks, i.e., the by-products of the
sub-network evaluation. Next, we reuse EKG to provide a gentle and informative
guidance to the pruned network while fine-tuning the pruned network. Since EKG
is implemented as a memory bank in both phases, it requires a negligible cost.
For example, when pruning and training ResNet-50, just 315 GPU hours are
required to remove around 45.04% of FLOPS without any performance degradation,
which can operate even on a low-spec workstation. the implemented code is
available at https://github.com/sseung0703/EKG.
