  In this paper, we address a recent trend in robotic home appliances to
include vision systems on personal devices, capable of personalizing the
appliances on the fly. In particular, we formulate and address an important
technical task of personal object search, which involves localization and
identification of personal items of interest on images captured by robotic
appliances, with each item referenced only by a few annotated images. The task
is crucial for robotic home appliances and mobile systems, which need to
process personal visual scenes or to operate with particular personal objects
(e.g., for grasping or navigation). In practice, personal object search
presents two main technical challenges. First, a robot vision system needs to
be able to distinguish between many fine-grained classes, in the presence of
occlusions and clutter. Second, the strict resource requirements for the
on-device system restrict the usage of most state-of-the-art methods for
few-shot learning and often prevent on-device adaptation. In this work, we
propose Swiss DINO: a simple yet effective framework for one-shot personal
object search based on the recent DINOv2 transformer model, which was shown to
have strong zero-shot generalization properties. Swiss DINO handles challenging
on-device personalized scene understanding requirements and does not require
any adaptation training. We show significant improvement (up to 55%) in
segmentation and recognition accuracy compared to the common lightweight
solutions, and significant footprint reduction of backbone inference time (up
to 100x) and GPU consumption (up to 10x) compared to the heavy
transformer-based solutions.
