  This paper introduces Honor of Kings Arena, a reinforcement learning (RL)
environment based on Honor of Kings, one of the world's most popular games at
present. Compared to other environments studied in most previous work, ours
presents new generalization challenges for competitive reinforcement learning.
It is a multi-agent problem with one agent competing against its opponent; and
it requires the generalization ability as it has diverse targets to control and
diverse opponents to compete with. We describe the observation, action, and
reward specifications for the Honor of Kings domain and provide an open-source
Python-based interface for communicating with the game engine. We provide
twenty target heroes with a variety of tasks in Honor of Kings Arena and
present initial baseline results for RL-based methods with feasible computing
resources. Finally, we showcase the generalization challenges imposed by Honor
of Kings Arena and possible remedies to the challenges. All of the software,
including the environment-class, are publicly available at
https://github.com/tencent-ailab/hok_env . The documentation is available at
https://aiarena.tencent.com/hok/doc/ .
