  In this paper, we present Tianshou, a highly modularized Python library for
deep reinforcement learning (DRL) that uses PyTorch as its backend. Tianshou
intends to be research-friendly by providing a flexible and reliable
infrastructure of DRL algorithms. It supports online and offline training with
more than 20 classic algorithms through a unified interface. To facilitate
related research and prove Tianshou's reliability, we have released Tianshou's
benchmark of MuJoCo environments, covering eight classic algorithms with
state-of-the-art performance. We open-sourced Tianshou at
https://github.com/thu-ml/tianshou/.
