  In this work, we introduce the task of 3D object instance re-localization
(RIO): given one or multiple objects in an RGB-D scan, we want to estimate
their corresponding 6DoF poses in another 3D scan of the same environment taken
at a later point in time. We consider RIO a particularly important task in 3D
vision since it enables a wide range of practical applications, including
AI-assistants or robots that are asked to find a specific object in a 3D scene.
To address this problem, we first introduce 3RScan, a novel dataset and
benchmark, which features 1482 RGB-D scans of 478 environments across multiple
time steps. Each scene includes several objects whose positions change over
time, together with ground truth annotations of object instances and their
respective 6DoF mappings among re-scans. Automatically finding 6DoF object
poses leads to a particular challenging feature matching task due to varying
partial observations and changes in the surrounding context. To this end, we
introduce a new data-driven approach that efficiently finds matching features
using a fully-convolutional 3D correspondence network operating on multiple
spatial scales. Combined with a 6DoF pose optimization, our method outperforms
state-of-the-art baselines on our newly-established benchmark, achieving an
accuracy of 30.58%.
