  In this paper, we present an end-to-end video compression network for P-frame
challenge on CLIC. We focus on deep neural network (DNN) based video
compression, and improve the current frameworks from three aspects. First, we
notice that pixel space residuals is sensitive to the prediction errors of
optical flow based motion compensation. To suppress the relative influence, we
propose to compress the residuals of image feature rather than the residuals of
image pixels. Furthermore, we combine the advantages of both pixel-level and
feature-level residual compression methods by model ensembling. Finally, we
propose a step-by-step training strategy to improve the training efficiency of
the whole framework. Experiment results indicate that our proposed method
achieves 0.9968 MS-SSIM on CLIC validation set and 0.9967 MS-SSIM on test set.
