  Weight play an essential role in deep learning network models. Unlike network
structure design, this article proposes the concept of weight augmentation,
focusing on weight exploration. The core of Weight Augmentation Strategy (WAS)
is to adopt random transformed weight coefficients training and transformed
coefficients, named Shadow Weight(SW), for networks that can be used to
calculate loss function to affect parameter updates. However, stochastic
gradient descent is applied to Plain Weight(PW), which is referred to as the
original weight of the network before the random transformation. During
training, numerous SW collectively form high-dimensional space, while PW is
directly learned from the distribution of SW instead of the data. The weight of
the accuracy-oriented mode(AOM) relies on PW, which guarantees the network is
highly robust and accurate. The desire-oriented mode(DOM) weight uses SW, which
is determined by the network model's unique functions based on WAT's
performance desires, such as lower computational complexity, lower sensitivity
to particular data, etc. The dual mode be switched at anytime if needed. WAT
extends the augmentation technique from data augmentation to weight, and it is
easy to understand and implement, but it can improve almost all networks
amazingly. Our experimental results show that convolutional neural networks,
such as VGG-16, ResNet-18, ResNet-34, GoogleNet, MobilementV2, and
Efficientment-Lite, can benefit much at little or no cost. The accuracy of
models is on the CIFAR100 and CIFAR10 datasets, which can be evaluated to
increase by 7.32\% and 9.28\%, respectively, with the highest values being
13.42\% and 18.93\%, respectively. In addition, DOM can reduce floating point
operations (FLOPs) by up to 36.33\%. The code is available at
https://github.com/zlearh/Weight-Augmentation-Technology.
