  Despite the recent advances in open-domain dialogue systems, building a
reliable evaluation metric is still a challenging problem. Recent studies
proposed learnable metrics based on classification models trained to
distinguish the correct response. However, neural classifiers are known to make
overly confident predictions for examples from unseen distributions. We propose
DEnsity, which evaluates a response by utilizing density estimation on the
feature space derived from a neural classifier. Our metric measures how likely
a response would appear in the distribution of human conversations. Moreover,
to improve the performance of DEnsity, we utilize contrastive learning to
further compress the feature space. Experiments on multiple response evaluation
datasets show that DEnsity correlates better with human evaluations than the
existing metrics. Our code is available at https://github.com/ddehun/DEnsity.
