  DNNs have been quickly and broadly exploited to improve the data analysis
quality in many complex science and engineering applications. Today's DNNs are
becoming deeper and wider because of increasing demand on the analysis quality
and more and more complex applications to resolve. The wide and deep DNNs,
however, require large amounts of resources, significantly restricting their
utilization on resource-constrained systems. Although some network
simplification methods have been proposed to address this issue, they suffer
from either low compression ratios or high compression errors, which may
introduce a costly retraining process for the target accuracy. In this paper,
we propose DeepSZ: an accuracy-loss bounded neural network compression
framework, which involves four key steps: network pruning, error bound
assessment, optimization for error bound configuration, and compressed model
generation, featuring a high compression ratio and low encoding time. The
contribution is three-fold. (1) We develop an adaptive approach to select the
feasible error bounds for each layer. (2) We build a model to estimate the
overall loss of accuracy based on the accuracy degradation caused by individual
decompressed layers. (3) We develop an efficient optimization algorithm to
determine the best-fit configuration of error bounds in order to maximize the
compression ratio under the user-set accuracy constraint. Experiments show that
DeepSZ can compress AlexNet and VGG-16 on the ImageNet by a compression ratio
of 46X and 116X, respectively, and compress LeNet-300-100 and LeNet-5 on the
MNIST by a compression ratio of 57X and 56X, respectively, with only up to 0.3%
loss of accuracy. Compared with other state-of-the-art methods, DeepSZ can
improve the compression ratio by up to 1.43X, the DNN encoding performance by
up to 4.0X (with four Nvidia Tesla V100 GPUs), and the decoding performance by
up to 6.2X.
