  In zero-shot image retrieval (ZSIR) task, embedding learning becomes more
attractive, however, many methods follow the traditional metric learning idea
and omit the problems behind zero-shot settings. In this paper, we first
emphasize the importance of learning visual discriminative metric and
preventing the partial/selective learning behavior of learner in ZSIR, and then
propose the Decoupled Metric Learning (DeML) framework to achieve these
individually. Instead of coarsely optimizing an unified metric, we decouple it
into multiple attention-specific parts so as to recurrently induce the
discrimination and explicitly enhance the generalization. And they are mainly
achieved by our object-attention module based on random walk graph propagation
and the channel-attention module based on the adversary constraint,
respectively. We demonstrate the necessity of addressing the vital problems in
ZSIR on the popular benchmarks, outperforming the state-of-theart methods by a
significant margin. Code is available at http://www.bhchen.cn
