  Domain divergence plays a significant role in estimating the performance of a
model in new domains. While there is a significant literature on divergence
measures, researchers find it hard to choose an appropriate divergence for a
given NLP application. We address this shortcoming by both surveying the
literature and through an empirical study. We develop a taxonomy of divergence
measures consisting of three classes -- Information-theoretic, Geometric, and
Higher-order measures and identify the relationships between them. Further, to
understand the common use-cases of these measures, we recognise three novel
applications -- 1) Data Selection, 2) Learning Representation, and 3) Decisions
in the Wild -- and use it to organise our literature. From this, we identify
that Information-theoretic measures are prevalent for 1) and 3), and
Higher-order measures are more common for 2). To further help researchers
choose appropriate measures to predict drop in performance -- an important
aspect of Decisions in the Wild, we perform correlation analysis spanning 130
domain adaptation scenarios, 3 varied NLP tasks and 12 divergence measures
identified from our survey. To calculate these divergences, we consider the
current contextual word representations (CWR) and contrast with the older
distributed representations. We find that traditional measures over word
distributions still serve as strong baselines, while higher-order measures with
CWR are effective.
