  Operators devoid of multiplication, such as Shift and Add, have gained
prominence for their compatibility with hardware. However, neural networks
(NNs) employing these operators typically exhibit lower accuracy compared to
conventional NNs with identical structures. ShiftAddAug uses costly
multiplication to augment efficient but less powerful multiplication-free
operators, improving performance without any inference overhead. It puts a
ShiftAdd tiny NN into a large multiplicative model and encourages it to be
trained as a sub-model to obtain additional supervision. In order to solve the
weight discrepancy problem between hybrid operators, a new weight sharing
method is proposed. Additionally, a novel two stage neural architecture search
is used to obtain better augmentation effects for smaller but stronger
multiplication-free tiny neural networks. The superiority of ShiftAddAug is
validated through experiments in image classification and semantic
segmentation, consistently delivering noteworthy enhancements. Remarkably, it
secures up to a 4.95% increase in accuracy on the CIFAR100 compared to its
directly trained counterparts, even surpassing the performance of
multiplicative NNs.
