  Recently, probabilistic predictive coding that directly models the
conditional distribution of latent features across successive frames for
temporal redundancy removal has yielded promising results. Existing methods
using a single-scale Variational AutoEncoder (VAE) must devise complex networks
for conditional probability estimation in latent space, neglecting multiscale
characteristics of video frames. Instead, this work proposes hierarchical
probabilistic predictive coding, for which hierarchal VAEs are carefully
designed to characterize multiscale latent features as a family of flexible
priors and posteriors to predict the probabilities of future frames. Under such
a hierarchical structure, lightweight networks are sufficient for prediction.
The proposed method outperforms representative learned video compression models
on common testing videos and demonstrates computational friendliness with much
less memory footprint and faster encoding/decoding. Extensive experiments on
adaptation to temporal patterns also indicate the better generalization of our
hierarchical predictive mechanism. Furthermore, our solution is the first to
enable progressive decoding that is favored in networked video applications
with packet loss.
