  Estimating the generalization performance is practically challenging on
out-of-distribution (OOD) data without ground-truth labels. While previous
methods emphasize the connection between distribution difference and OOD
accuracy, we show that a large domain gap not necessarily leads to a low test
accuracy. In this paper, we investigate this problem from the perspective of
feature separability empirically and theoretically. Specifically, we propose a
dataset-level score based upon feature dispersion to estimate the test accuracy
under distribution shift. Our method is inspired by desirable properties of
features in representation learning: high inter-class dispersion and high
intra-class compactness. Our analysis shows that inter-class dispersion is
strongly correlated with the model accuracy, while intra-class compactness does
not reflect the generalization performance on OOD data. Extensive experiments
demonstrate the superiority of our method in both prediction performance and
computational efficiency.
