  We introduce AARGH, an end-to-end task-oriented dialog system combining
retrieval and generative approaches in a single model, aiming at improving
dialog management and lexical diversity of outputs. The model features a new
response selection method based on an action-aware training objective and a
simplified single-encoder retrieval architecture which allow us to build an
end-to-end retrieval-enhanced generation model where retrieval and generation
share most of the parameters. On the MultiWOZ dataset, we show that our
approach produces more diverse outputs while maintaining or improving state
tracking and context-to-response generation performance, compared to
state-of-the-art baselines.
