  Real Time Strategy (RTS) games require macro strategies as well as micro
strategies to obtain satisfactory performance since it has large state space,
action space, and hidden information. This paper presents a novel hierarchical
reinforcement learning model for mastering Multiplayer Online Battle Arena
(MOBA) games, a sub-genre of RTS games. The novelty of this work are: (1)
proposing a hierarchical framework, where agents execute macro strategies by
imitation learning and carry out micromanipulations through reinforcement
learning, (2) developing a simple self-learning method to get better sample
efficiency for training, and (3) designing a dense reward function for
multi-agent cooperation in the absence of game engine or Application
Programming Interface (API). Finally, various experiments have been performed
to validate the superior performance of the proposed method over other
state-of-the-art reinforcement learning algorithms. Agent successfully learns
to combat and defeat bronze-level built-in AI with 100% win rate, and
experiments show that our method can create a competitive multi-agent for a
kind of mobile MOBA game {\it King of Glory} in 5v5 mode.
