  An intuitive way to search for images is to use queries composed of an
example image and a complementary text. While the first provides rich and
implicit context for the search, the latter explicitly calls for new traits, or
specifies how some elements of the example image should be changed to retrieve
the desired target image. Current approaches typically combine the features of
each of the two elements of the query into a single representation, which can
then be compared to the ones of the potential target images. Our work aims at
shedding new light on the task by looking at it through the prism of two
familiar and related frameworks: text-to-image and image-to-image retrieval.
Taking inspiration from them, we exploit the specific relation of each query
element with the targeted image and derive light-weight attention mechanisms
which enable to mediate between the two complementary modalities. We validate
our approach on several retrieval benchmarks, querying with images and their
associated free-form text modifiers. Our method obtains state-of-the-art
results without resorting to side information, multi-level features, heavy
pre-training nor large architectures as in previous works.
