  Instructions-tuned Large Language Models (LLMs) gained recently huge
popularity thanks to their ability to interact with users through conversation.
In this work we aim to evaluate their ability to complete multi-turn tasks and
interact with external databases in the context of established task-oriented
dialogue benchmarks. We show that for explicit belief state tracking, LLMs
underperform compared to specialized task-specific models. Nevertheless, they
show ability to guide the dialogue to successful ending if given correct slot
values. Furthermore this ability improves with access to true belief state
distribution or in-domain examples.
