  Whitening loss offers a theoretical guarantee against feature collapse in
self-supervised learning (SSL) with joint embedding architectures. Typically,
it involves a hard whitening approach, transforming the embedding and applying
loss to the whitened output. In this work, we introduce Spectral Transformation
(ST), a framework to modulate the spectrum of embedding and to seek for
functions beyond whitening that can avoid dimensional collapse. We show that
whitening is a special instance of ST by definition, and our empirical
investigations unveil other ST instances capable of preventing collapse.
Additionally, we propose a novel ST instance named IterNorm with trace loss
(INTL). Theoretical analysis confirms INTL's efficacy in preventing collapse
and modulating the spectrum of embedding toward equal-eigenvalues during
optimization. Our experiments on ImageNet classification and COCO object
detection demonstrate INTL's potential in learning superior representations.
The code is available at https://github.com/winci-ai/INTL.
