  Inspired by the recent advances of image super-resolution using convolutional
neural network (CNN), we propose a CNN-based block up-sampling scheme for intra
frame coding. A block can be down-sampled before being compressed by normal
intra coding, and then up-sampled to its original resolution. Different from
previous studies on down/up-sampling-based coding, the up-sampling methods in
our scheme have been designed by training CNN instead of hand-crafted. We
explore a new CNN structure for up-sampling, which features deconvolution of
feature maps, multi-scale fusion, and residue learning, making the network both
compact and efficient. We also design different networks for the up-sampling of
luma and chroma components, respectively, where the chroma up-sampling CNN
utilizes the luma information to boost its performance. In addition, we design
a two-stage up-sampling process, the first stage being within the
block-by-block coding loop, and the second stage being performed on the entire
frame, so as to refine block boundaries. We also empirically study how to set
the coding parameters of down-sampled blocks for pursuing the frame-level
rate-distortion optimization. Our proposed scheme is implemented into the High
Efficiency Video Coding (HEVC) reference software, and a comprehensive set of
experiments have been performed to evaluate our methods. Experimental results
show that our scheme achieves significant bits saving compared with HEVC anchor
especially at low bit rates, leading to on average 5.5% BD-rate reduction on
common test sequences and on average 9.0% BD-rate reduction on ultra high
definition (UHD) test sequences.
