  Client-side video players employ adaptive bitrate (ABR) algorithms to
optimize user quality of experience (QoE). We evaluate recently proposed
RL-based ABR methods in Facebook's web-based video streaming platform.
Real-world ABR contains several challenges that requires customized designs
beyond off-the-shelf RL algorithms -- we implement a scalable neural network
architecture that supports videos with arbitrary bitrate encodings; we design a
training method to cope with the variance resulting from the stochasticity in
network conditions; and we leverage constrained Bayesian optimization for
reward shaping in order to optimize the conflicting QoE objectives. In a
week-long worldwide deployment with more than 30 million video streaming
sessions, our RL approach outperforms the existing human-engineered ABR
algorithms.
