  One of the difficulties in training dialogue systems is the lack of training
data. We explore the possibility of creating dialogue data through the
interaction between a dialogue system and a user simulator. Our goal is to
develop a modelling framework that can incorporate new dialogue scenarios
through self-play between the two agents. In this framework, we first pre-train
the two agents on a collection of source domain dialogues, which equips the
agents to converse with each other via natural language. With further
fine-tuning on a small amount of target domain data, the agents continue to
interact with the aim of improving their behaviors using reinforcement learning
with structured reward functions. In experiments on the MultiWOZ dataset, two
practical transfer learning problems are investigated: 1) domain adaptation and
2) single-to-multiple domain transfer. We demonstrate that the proposed
framework is highly effective in bootstrapping the performance of the two
agents in transfer learning. We also show that our method leads to improvements
in dialogue system performance on complete datasets.
