  The ability to accurately track what happens during a conversation is
essential for the performance of a dialogue system. Current state-of-the-art
multi-domain dialogue state trackers achieve just over 55% accuracy on the
current go-to benchmark, which means that in almost every second dialogue turn
they place full confidence in an incorrect dialogue state. Belief trackers, on
the other hand, maintain a distribution over possible dialogue states. However,
they lack in performance compared to dialogue state trackers, and do not
produce well calibrated distributions. In this work we present state-of-the-art
performance in calibration for multi-domain dialogue belief trackers using a
calibrated ensemble of models. Our resulting dialogue belief tracker also
outperforms previous dialogue belief tracking models in terms of accuracy.
