  Dialogue datasets are crucial for deep learning-based task-oriented dialogue
system research. While numerous English language multi-domain task-oriented
dialogue datasets have been developed and contributed to significant
advancements in task-oriented dialogue systems, such a dataset does not exist
in Japanese, and research in this area is limited compared to that in English.
In this study, towards the advancement of research and development of
task-oriented dialogue systems in Japanese, we constructed JMultiWOZ, the first
Japanese language large-scale multi-domain task-oriented dialogue dataset.
Using JMultiWOZ, we evaluated the dialogue state tracking and response
generation capabilities of the state-of-the-art methods on the existing major
English benchmark dataset MultiWOZ2.2 and the latest large language model
(LLM)-based methods. Our evaluation results demonstrated that JMultiWOZ
provides a benchmark that is on par with MultiWOZ2.2. In addition, through
evaluation experiments of interactive dialogues with the models and human
participants, we identified limitations in the task completion capabilities of
LLMs in Japanese.
