  MultiWOZ 2.0 (Budzianowski et al., 2018) is a recently released multi-domain
dialogue dataset spanning 7 distinct domains and containing over 10,000
dialogues. Though immensely useful and one of the largest resources of its kind
to-date, MultiWOZ 2.0 has a few shortcomings. Firstly, there is substantial
noise in the dialogue state annotations and dialogue utterances which
negatively impact the performance of state-tracking models. Secondly, follow-up
work (Lee et al., 2019) has augmented the original dataset with user dialogue
acts. This leads to multiple co-existent versions of the same dataset with
minor modifications. In this work we tackle the aforementioned issues by
introducing MultiWOZ 2.1. To fix the noisy state annotations, we use
crowdsourced workers to re-annotate state and utterances based on the original
utterances in the dataset. This correction process results in changes to over
32% of state annotations across 40% of the dialogue turns. In addition, we fix
146 dialogue utterances by canonicalizing slot values in the utterances to the
values in the dataset ontology. To address the second problem, we combined the
contributions of the follow-up works into MultiWOZ 2.1. Hence, our dataset also
includes user dialogue acts as well as multiple slot descriptions per dialogue
state slot. We then benchmark a number of state-of-the-art dialogue state
tracking models on the MultiWOZ 2.1 dataset and show the joint state tracking
performance on the corrected state annotations. We are publicly releasing
MultiWOZ 2.1 to the community, hoping that this dataset resource will allow for
more effective models across various dialogue subproblems to be built in the
future.
