  Interest in unsupervised domain adaptation (UDA) has surged in recent years,
resulting in a plethora of new algorithms. However, as is often the case in
fast-moving fields, baseline algorithms are not tested to the extent that they
should be. Furthermore, little attention has been paid to validation methods,
i.e. the methods for estimating the accuracy of a model in the absence of
target domain labels. This is despite the fact that validation methods are a
crucial component of any UDA train/val pipeline. In this paper, we show via
large-scale experimentation that 1) in the oracle setting, the difference in
accuracy between UDA algorithms is smaller than previously thought, 2)
state-of-the-art validation methods are not well-correlated with accuracy, and
3) differences between UDA algorithms are dwarfed by the drop in accuracy
caused by validation methods.
