  Unsupervised learning has recently significantly gained in popularity,
especially with deep learning-based approaches. Despite numerous successes and
approaching supervised-level performance on a variety of academic benchmarks,
it is still hard to train and evaluate SSL models in practice due to the
unsupervised nature of the problem. Even with networks trained in a supervised
fashion, it is often unclear whether they will perform well when transferred to
another domain.
  Past works are generally limited to assessing the amount of information
contained in embeddings, which is most relevant for self-supervised learning of
deep neural networks. This works chooses to follow a different approach: can we
quantify how easy it is to linearly separate the data in a stable way? We
survey the literature and uncover three methods that could be potentially used
for evaluating quality of representations. We also introduce one novel method
based on recent advances in understanding the high-dimensional geometric
structure of self-supervised learning.
  We conduct extensive experiments and study the properties of these metrics
and ones introduced in the previous work. Our results suggest that while there
is no free lunch, there are metrics that can robustly estimate embedding
quality in an unsupervised way.
