  How important is it for training and evaluation sets to not have class
overlap in image retrieval? We revisit Google Landmarks v2 clean, the most
popular training set, by identifying and removing class overlap with Revisited
Oxford and Paris [34], the most popular evaluation set. By comparing the
original and the new RGLDv2-clean on a benchmark of reproduced state-of-the-art
methods, our findings are striking. Not only is there a dramatic drop in
performance, but it is inconsistent across methods, changing the ranking.What
does it take to focus on objects or interest and ignore background clutter when
indexing? Do we need to train an object detector and the representation
separately? Do we need location supervision? We introduce Single-stage
Detect-to-Retrieve (CiDeR), an end-to-end, single-stage pipeline to detect
objects of interest and extract a global image representation. We outperform
previous state-of-the-art on both existing training sets and the new
RGLDv2-clean. Our dataset is available at
https://github.com/dealicious-inc/RGLDv2-clean.
