  Reinforcement learning has been widely adopted to model dialogue managers in
task-oriented dialogues. However, the user simulator provided by
state-of-the-art dialogue frameworks are only rough approximations of human
behaviour. The ability to learn from a small number of human interactions is
hence crucial, especially on multi-domain and multi-task environments where the
action space is large. We therefore propose to use structured policies to
improve sample efficiency when learning on these kinds of environments. We also
evaluate the impact of learning from human vs simulated experts. Among the
different levels of structure that we tested, the graph neural networks (GNNs)
show a remarkable superiority by reaching a success rate above 80% with only 50
dialogues, when learning from simulated experts. They also show superiority
when learning from human experts, although a performance drop was observed,
indicating a possible difficulty in capturing the variability of human
strategies. We therefore suggest to concentrate future research efforts on
bridging the gap between human data, simulators and automatic evaluators in
dialogue frameworks.
