  In this paper, we propose a Collaboration of Experts (CoE) framework to pool
together the expertise of multiple networks towards a common aim. Each expert
is an individual network with expertise on a unique portion of the dataset,
which enhances the collective capacity. Given a sample, an expert is selected
by the delegator, which simultaneously outputs a rough prediction to support
early termination. To fulfill this framework, we propose three modules to impel
each model to play its role, namely weight generation module (WGM), label
generation module (LGM) and variance calculation module (VCM). Our method
achieves the state-of-the-art performance on ImageNet, 80.7% top-1 accuracy
with 194M FLOPs. Combined with PWLU activation function and CondConv, CoE
further achieves the accuracy of 80.0% with only 100M FLOPs for the first time.
More importantly, our method is hardware friendly and achieves a 3-6x speedup
compared with some existing conditional computation approaches.
