  Video streaming usage has seen a significant rise as entertainment,
education, and business increasingly rely on online video. Optimizing video
compression has the potential to increase access and quality of content to
users, and reduce energy use and costs overall. In this paper, we present an
application of the MuZero algorithm to the challenge of video compression.
Specifically, we target the problem of learning a rate control policy to select
the quantization parameters (QP) in the encoding process of libvpx, an open
source VP9 video compression library widely used by popular video-on-demand
(VOD) services. We treat this as a sequential decision making problem to
maximize the video quality with an episodic constraint imposed by the target
bitrate. Notably, we introduce a novel self-competition based reward mechanism
to solve constrained RL with variable constraint satisfaction difficulty, which
is challenging for existing constrained RL methods. We demonstrate that the
MuZero-based rate control achieves an average 6.28% reduction in size of the
compressed videos for the same delivered video quality level (measured as PSNR
BD-rate) compared to libvpx's two-pass VBR rate control policy, while having
better constraint satisfaction behavior.
