  Task-oriented dialog systems are often trained on human/human dialogs, such
as collected from Wizard-of-Oz interfaces. However, human/human corpora are
frequently too small for supervised training to be effective. This paper
investigates two approaches to training agent-bots and user-bots through
self-play, in which they autonomously explore an API environment, discovering
communication strategies that enable them to solve the task. We give empirical
results for both reinforcement learning and game-theoretic equilibrium finding.
