  Content-based image retrieval has seen astonishing progress over the past
decade, especially for the task of retrieving images of the same object that is
depicted in the query image. This scenario is called instance or object
retrieval and requires matching fine-grained visual patterns between images.
Semantics, however, do not play a crucial role. This brings rise to the
question: Do the recent advances in instance retrieval transfer to more generic
image retrieval scenarios? To answer this question, we first provide a brief
overview of the most relevant milestones of instance retrieval. We then apply
them to a semantic image retrieval task and find that they perform inferior to
much less sophisticated and more generic methods in a setting that requires
image understanding. Following this, we review existing approaches to closing
this so-called semantic gap by integrating prior world knowledge. We conclude
that the key problem for the further advancement of semantic image retrieval
lies in the lack of a standardized task definition and an appropriate benchmark
dataset.
