  Recent dialogue datasets like MultiWOZ 2.1 and Taskmaster-1 constitute some
of the most challenging tasks for present-day dialogue models and, therefore,
are widely used for system evaluation. We identify several issues with the
above-mentioned datasets, such as history independence, strong knowledge base
dependence, and ambiguous system responses. Finally, we outline key desiderata
for future datasets that we believe would be more suitable for the construction
of conversational artificial intelligence.
