  This paper addresses the problem of set-to-set matching, which involves
matching two different sets of items based on some criteria, especially in the
case of high-dimensional items like images. Although neural networks have been
applied to solve this problem, most machine learning-based approaches assume
that the training and test data follow the same distribution, which is not
always true in real-world scenarios. To address this limitation, we introduce
SHIFT15M, a dataset that can be used to evaluate set-to-set matching models
when the distribution of data changes between training and testing. We conduct
benchmark experiments that demonstrate the performance drop of naive methods
due to distribution shift. Additionally, we provide software to handle the
SHIFT15M dataset in a simple manner, with the URL for the software to be made
available after publication of this manuscript. We believe proposed SHIFT15M
dataset provide a valuable resource for evaluating set-to-set matching models
under the distribution shift.
