  Reinforcement learning (RL) is a promising approach to solve dialogue policy
optimisation. Traditional RL algorithms, however, fail to scale to large
domains due to the curse of dimensionality. We propose a novel Dialogue
Management architecture, based on Feudal RL, which decomposes the decision into
two steps; a first step where a master policy selects a subset of primitive
actions, and a second step where a primitive action is chosen from the selected
subset. The structural information included in the domain ontology is used to
abstract the dialogue state space, taking the decisions at each step using
different parts of the abstracted state. This, combined with an information
sharing mechanism between slots, increases the scalability to large domains. We
show that an implementation of this approach, based on Deep-Q Networks,
significantly outperforms previous state of the art in several dialogue domains
and environments, without the need of any additional reward signal.
