  Consider a scenario where we are supplied with a number of ready-to-use
models trained on a certain source domain and hope to directly apply the most
appropriate ones to different target domains based on the models' relative
performance. Ideally we should annotate a validation set for model performance
assessment on each new target environment, but such annotations are often very
expensive. Under this circumstance, we introduce the problem of ranking models
in unlabeled new environments. For this problem, we propose to adopt a proxy
dataset that 1) is fully labeled and 2) well reflects the true model rankings
in a given target environment, and use the performance rankings on the proxy
sets as surrogates. We first select labeled datasets as the proxy.
Specifically, datasets that are more similar to the unlabeled target domain are
found to better preserve the relative performance rankings. Motivated by this,
we further propose to search the proxy set by sampling images from various
datasets that have similar distributions as the target. We analyze the problem
and its solutions on the person re-identification (re-ID) task, for which
sufficient datasets are publicly available, and show that a carefully
constructed proxy set effectively captures relative performance ranking in new
environments. Code is available at \url{https://github.com/sxzrt/Proxy-Set}.
