  In this paper, we study a new problem arising from the emerging MPEG
standardization effort Video Coding for Machine (VCM), which aims to bridge the
gap between visual feature compression and classical video coding. VCM is
committed to address the requirement of compact signal representation for both
machine and human vision in a more or less scalable way. To this end, we make
endeavors in leveraging the strength of predictive and generative models to
support advanced compression techniques for both machine and human vision tasks
simultaneously, in which visual features serve as a bridge to connect
signal-level and task-level compact representations in a scalable manner.
Specifically, we employ a conditional deep generation network to reconstruct
video frames with the guidance of learned motion pattern. By learning to
extract sparse motion pattern via a predictive model, the network elegantly
leverages the feature representation to generate the appearance of to-be-coded
frames via a generative model, relying on the appearance of the coded key
frames. Meanwhile, the sparse motion pattern is compact and highly effective
for high-level vision tasks, e.g. action recognition. Experimental results
demonstrate that our method yields much better reconstruction quality compared
with the traditional video codecs (0.0063 gain in SSIM), as well as
state-of-the-art action recognition performance over highly compressed videos
(9.4% gain in recognition accuracy), which showcases a promising paradigm of
coding signal for both human and machine vision.
