  Hardware support for deep convolutional neural networks (CNNs) is critical to
advanced computer vision in mobile and embedded devices. Current designs,
however, accelerate generic CNNs; they do not exploit the unique
characteristics of real-time vision. We propose to use the temporal redundancy
in natural video to avoid unnecessary computation on most frames. A new
algorithm, activation motion compensation, detects changes in the visual input
and incrementally updates a previously-computed output. The technique takes
inspiration from video compression and applies well-known motion estimation
techniques to adapt to visual changes. We use an adaptive key frame rate to
control the trade-off between efficiency and vision quality as the input
changes. We implement the technique in hardware as an extension to existing
state-of-the-art CNN accelerator designs. The new unit reduces the average
energy per frame by 54.2%, 61.7%, and 87.6% for three CNNs with less than 1%
loss in vision accuracy.
