  Dialogue policy training for composite tasks, such as restaurant reservation
in multiple places, is a practically important and challenging problem.
Recently, hierarchical deep reinforcement learning (HDRL) methods have achieved
good performance in composite tasks. However, in vanilla HDRL, both top-level
and low-level policies are all represented by multi-layer perceptrons (MLPs)
which take the concatenation of all observations from the environment as the
input for predicting actions. Thus, traditional HDRL approach often suffers
from low sampling efficiency and poor transferability. In this paper, we
address these problems by utilizing the flexibility of graph neural networks
(GNNs). A novel ComNet is proposed to model the structure of a hierarchical
agent. The performance of ComNet is tested on composited tasks of the PyDial
benchmark. Experiments show that ComNet outperforms vanilla HDRL systems with
performance close to the upper bound. It not only achieves sample efficiency
but also is more robust to noise while maintaining the transferability to other
composite tasks.
