  Quantization has become increasingly pivotal in addressing the steadily
increasing computational and memory requirements of Deep Neural Networks
(DNNs). By reducing the number of bits used to represent weights and
activations (typically from 32-bit floating-point to 16-bit or 8-bit integers),
quantization reduces the memory footprint, energy consumption, and execution
time of DNN models. However, traditional quantization methods typically focus
on the inference of DNNs, while the training process still relies on
floating-point operations. To date, only one work in the literature has
addressed integer-only training for Multi-Layer Perceptron (MLP) architectures.
This work introduces NITRO-D, a new framework for training arbitrarily deep
integer-only Convolutional Neural Networks (CNNs) that operate entirely< in the
integer-only domain for both training and inference. NITRO-D is the first
framework in the literature enabling the training of integer-only CNNs without
the need to introduce a quantization scheme. Specifically, NITRO-D introduces a
novel architecture integrating multiple integer local-loss blocks, which
include the proposed NITRO Scaling Layer and the NITRO-ReLU activation
function. Additionally, it introduces a novel integer-only learning algorithm
derived from Local Error Signals (LES), utilizing IntegerSGD, an optimizer
specifically designed to operate in an integer-only context. NITRO-D is
implemented in an open-source Python library. Extensive experimental
evaluations demonstrate its effectiveness across several state-of-the-art image
recognition datasets. Results show significant performance improvements from
2.47% to 5.96% for integer-only MLP architectures over the state-of-the-art
solution, and the capability of training integer-only CNN architectures with
minimal accuracy degradation from -0.15% to -4.22% compared to floating-point
LES.
