  In this work, we present BasisNet which combines recent advancements in
efficient neural network architectures, conditional computation, and early
termination in a simple new form. Our approach incorporates a lightweight model
to preview the input and generate input-dependent combination coefficients,
which later controls the synthesis of a more accurate specialist model to make
final prediction. The two-stage model synthesis strategy can be applied to any
network architectures and both stages are jointly trained. We also show that
proper training recipes are critical for increasing generalizability for such
high capacity neural networks. On ImageNet classification benchmark, our
BasisNet with MobileNets as backbone demonstrated clear advantage on
accuracy-efficiency trade-off over several strong baselines. Specifically,
BasisNet-MobileNetV3 obtained 80.3% top-1 accuracy with only 290M Multiply-Add
operations, halving the computational cost of previous state-of-the-art without
sacrificing accuracy. With early termination, the average cost can be further
reduced to 198M MAdds while maintaining accuracy of 80.0% on ImageNet.
