  The ever-increasing computational complexity of deep learning models makes
their training and deployment difficult on various cloud and edge platforms.
Replacing floating-point arithmetic with low-bit integer arithmetic is a
promising approach to save energy, memory footprint, and latency of deep
learning models. As such, quantization has attracted the attention of
researchers in recent years. However, using integer numbers to form a fully
functional integer training pipeline including forward pass, back-propagation,
and stochastic gradient descent is not studied in detail. Our empirical and
mathematical results reveal that integer arithmetic seems to be enough to train
deep learning models. Unlike recent proposals, instead of quantization, we
directly switch the number representation of computations. Our novel training
method forms a fully integer training pipeline that does not change the
trajectory of the loss and accuracy compared to floating-point, nor does it
need any special hyper-parameter tuning, distribution adjustment, or gradient
clipping. Our experimental results show that our proposed method is effective
in a wide variety of tasks such as classification (including vision
transformers), object detection, and semantic segmentation.
