  Recently, there has been progress in supervised funetuning pretrained GPT-2
to build end-to-end task-oriented dialog (TOD) systems. However, online
reinforcement learning of a GPT-2 based dialog system (DS), together with a
end-to-end user simulator (US), has not ever been explored. Moreover, a
drawback with existing GPT-2 based TOD systems is that they mostly employ the
whole dialog history as input, which brings inefficiencies in memory and
compute. In this paper, we first propose Simplified Generative Architectures
(SGA) for DS and US respectively, both based on GPT-2 but using shortened
history. Then, we successfully develop Jointly Reinforced US and DS, called
SGA-JRUD. Our DS with the proposed SGA, when only supervised trained, achieves
state-of-the-art performance on MultiWOZ2.1 and is more compute-efficient in
both training and generation. Extensive experiments on MultiWOZ2.1 further show
the superiority of SGA-JRUD in both offline and online evaluations.
