  Recent advances in generalized image understanding have seen a surge in the
use of deep convolutional neural networks (CNN) across a broad range of
image-based detection, classification and prediction tasks. Whilst the reported
performance of these approaches is impressive, this study investigates the
hitherto unapproached question of the impact of commonplace image and video
compression techniques on the performance of such deep learning architectures.
Focusing on the JPEG and H.264 (MPEG-4 AVC) as a representative proxy for
contemporary lossy image/video compression techniques that are in common use
within network-connected image/video devices and infrastructure, we examine the
impact on performance across five discrete tasks: human pose estimation,
semantic segmentation, object detection, action recognition, and monocular
depth estimation. As such, within this study we include a variety of network
architectures and domains spanning end-to-end convolution, encoder-decoder,
region-based CNN (R-CNN), dual-stream, and generative adversarial networks
(GAN). Our results show a non-linear and non-uniform relationship between
network performance and the level of lossy compression applied. Notably,
performance decreases significantly below a JPEG quality (quantization) level
of 15% and a H.264 Constant Rate Factor (CRF) of 40. However, retraining said
architectures on pre-compressed imagery conversely recovers network performance
by up to 78.4% in some cases. Furthermore, there is a correlation between
architectures employing an encoder-decoder pipeline and those that demonstrate
resilience to lossy image compression. The characteristics of the relationship
between input compression to output task performance can be used to inform
design decisions within future image/video devices and infrastructure.
