  Recent ensemble learning methods for image classification have been shown to
improve classification accuracy with low extra cost. However, they still
require multiple trained models for ensemble inference, which eventually
becomes a significant burden when the model size increases. In this paper, we
propose a low-cost ensemble learning and inference, called Network Fission
Ensembles (NFE), by converting a conventional network itself into a multi-exit
structure. Starting from a given initial network, we first prune some of the
weights to reduce the training burden. We then group the remaining weights into
several sets and create multiple auxiliary paths using each set to construct
multi-exits. We call this process Network Fission. Through this, multiple
outputs can be obtained from a single network, which enables ensemble learning.
Since this process simply changes the existing network structure to multi-exits
without using additional networks, there is no extra computational burden for
ensemble learning and inference. Moreover, by learning from multiple losses of
all exits, the multi-exits improve performance via regularization, and high
performance can be achieved even with increased network sparsity. With our
simple yet effective method, we achieve significant improvement compared to
existing ensemble methods. The code is available at
https://github.com/hjdw2/NFE.
