  With the widespread use of installed cameras, video-based monitoring
approaches have seized considerable attention for different purposes like
assisted living. Temporal redundancy and the sheer size of raw videos are the
two most common problematic issues related to video processing algorithms. Most
of the existing methods mainly focused on increasing accuracy by exploring
consecutive frames, which is laborious and cannot be considered for real-time
applications. Since videos are mostly stored and transmitted in compressed
format, these kinds of videos are available on many devices. Compressed videos
contain a multitude of beneficial information, such as motion vectors and
quantized coefficients. Proper use of this available information can greatly
improve the video understanding methods' performance. This paper presents an
approach for using residual data, available in compressed videos directly,
which can be obtained by a light partially decoding procedure. In addition, a
method for accumulating similar residuals is proposed, which dramatically
reduces the number of processed frames for action recognition. Applying neural
networks exclusively for accumulated residuals in the compressed domain
accelerates performance, while the classification results are highly
competitive with raw video approaches.
