  This work aims to assess how well a model performs under distribution shifts
without using labels. While recent methods study prediction confidence, this
work reports prediction dispersity is another informative cue. Confidence
reflects whether the individual prediction is certain; dispersity indicates how
the overall predictions are distributed across all categories. Our key insight
is that a well-performing model should give predictions with high confidence
and high dispersity. That is, we need to consider both properties so as to make
more accurate estimates. To this end, we use the nuclear norm that has been
shown to be effective in characterizing both properties. Extensive experiments
validate the effectiveness of nuclear norm for various models (e.g., ViT and
ConvNeXt), different datasets (e.g., ImageNet and CUB-200), and diverse types
of distribution shifts (e.g., style shift and reproduction shift). We show that
the nuclear norm is more accurate and robust in accuracy estimation than
existing methods. Furthermore, we validate the feasibility of other
measurements (e.g., mutual information maximization) for characterizing
dispersity and confidence. Lastly, we investigate the limitation of the nuclear
norm, study its improved variant under severe class imbalance, and discuss
potential directions.
