  The image-text retrieval task aims to retrieve relevant information from a
given image or text. The main challenge is to unify multimodal representation
and distinguish fine-grained differences across modalities, thereby finding
similar contents and filtering irrelevant contents. However, existing methods
mainly focus on unified semantic representation and concept alignment for
multi-modalities, while the fine-grained differences across modalities have
rarely been studied before, making it difficult to solve the information
asymmetry problem. In this paper, we propose a novel asymmetry-sensitive
contrastive learning method. By generating corresponding positive and negative
samples for different asymmetry types, our method can simultaneously ensure
fine-grained semantic differentiation and unified semantic representation
between multi-modalities. Additionally, a hierarchical cross-modal fusion
method is proposed, which integrates global and local-level features through a
multimodal attention mechanism to achieve concept alignment. Extensive
experiments performed on MSCOCO and Flickr30K, demonstrate the effectiveness
and superiority of our proposed method.
