  Dialogue state tracking (DST) aims to extract essential information from
multi-turn dialogue situations and take appropriate actions. A belief state,
one of the core pieces of information, refers to the subject and its specific
content, and appears in the form of domain-slot-value. The trained model
predicts "accumulated" belief states in every turn, and joint goal accuracy and
slot accuracy are mainly used to evaluate the prediction; however, we specify
that the current evaluation metrics have a critical limitation when evaluating
belief states accumulated as the dialogue proceeds, especially in the most used
MultiWOZ dataset. Additionally, we propose relative slot accuracy to complement
existing metrics. Relative slot accuracy does not depend on the number of
predefined slots, and allows intuitive evaluation by assigning relative scores
according to the turn of each dialogue. This study also encourages not solely
the reporting of joint goal accuracy, but also various complementary metrics in
DST tasks for the sake of a realistic evaluation.
