  Taobao Search consists of two phases: the retrieval phase and the ranking
phase. Given a user query, the retrieval phase returns a subset of candidate
products for the following ranking phase. Recently, the paradigm of
pre-training and fine-tuning has shown its potential in incorporating visual
clues into retrieval tasks. In this paper, we focus on solving the problem of
text-to-multimodal retrieval in Taobao Search. We consider that users'
attention on titles or images varies on products. Hence, we propose a novel
Modal Adaptation module for cross-modal fusion, which helps assigns appropriate
weights on texts and images across products. Furthermore, in e-commerce search,
user queries tend to be brief and thus lead to significant semantic imbalance
between user queries and product titles. Therefore, we design a separate text
encoder and a Keyword Enhancement mechanism to enrich the query representations
and improve text-to-multimodal matching. To this end, we present a novel
vision-language (V+L) pre-training methods to exploit the multimodal
information of (user query, product title, product image). Extensive
experiments demonstrate that our retrieval-specific pre-training model
(referred to as MAKE) outperforms existing V+L pre-training methods on the
text-to-multimodal retrieval task. MAKE has been deployed online and brings
major improvements on the retrieval system of Taobao Search.
