  The neural collapse (NC) phenomenon describes an underlying geometric
symmetry for deep neural networks, where both deeply learned features and
classifiers converge to a simplex equiangular tight frame. It has been shown
that both cross-entropy loss and mean square error can provably lead to NC. We
remove NC's key assumption on the feature dimension and the number of classes,
and then present a generalized neural collapse (GNC) hypothesis that
effectively subsumes the original NC. Inspired by how NC characterizes the
training target of neural networks, we decouple GNC into two objectives:
minimal intra-class variability and maximal inter-class separability. We then
use hyperspherical uniformity (which characterizes the degree of uniformity on
the unit hypersphere) as a unified framework to quantify these two objectives.
Finally, we propose a general objective -- hyperspherical uniformity gap (HUG),
which is defined by the difference between inter-class and intra-class
hyperspherical uniformity. HUG not only provably converges to GNC, but also
decouples GNC into two separate objectives. Unlike cross-entropy loss that
couples intra-class compactness and inter-class separability, HUG enjoys more
flexibility and serves as a good alternative loss function. Empirical results
show that HUG works well in terms of generalization and robustness.
