  The concepts of overfitting and generalization are vital for evaluating
machine learning models. In this work, we show that the popular Recall@K metric
depends on the number of classes in the dataset, which limits its ability to
estimate generalization. To fix this issue, we propose a new metric, which
measures retrieval performance, and, unlike Recall@K, estimates generalization.
We apply the proposed metric to popular image retrieval methods and provide new
insights about deep metric learning generalization.
