  Sentence embedding is a cornerstone in NLP. Whitening has been claimed to be
an effective operation to improve embedding quality obtained from Large
Language Models (LLMs). However, we find that the efficacy of whitening is
model-dependent and task-dependent. In particular, whitening degenerates
embeddings for classification tasks. The conclusion is supported by extensive
experiments. We also explored a variety of whitening operations, including PCA,
ZCA, PCA-Cor, ZCA-Cor and Cholesky whitenings. A by-product of our research is
embedding evaluation platform for LLMs called SentEval+.
