  It has been arduous to assess the progress of a policy learning algorithm in
the domain of hierarchical task with high dimensional action space due to the
lack of a commonly accepted benchmark. In this work, we propose a new
light-weight benchmark task called Diner Dash for evaluating the performance in
a complicated task with high dimensional action space. In contrast to the
traditional Atari games that only have a flat structure of goals and very few
actions, the proposed benchmark task has a hierarchical task structure and size
of 57 for the action space and hence can facilitate the development of policy
learning in complicated tasks. On top of that, we introduce Decomposed Policy
Graph Modelling (DPGM), an algorithm that combines both graph modelling and
deep learning to allow explicit domain knowledge embedding and achieves
significant improvement comparing to the baseline. In the experiments, we have
shown the effectiveness of the domain knowledge injection via a specially
designed imitation algorithm as well as results of other popular algorithms.
