  In recent years, Cross-Modal Hashing (CMH) has aroused much attention due to
its fast query speed and efficient storage. Previous literatures have achieved
promising results for Cross-Modal Retrieval (CMR) by discovering discriminative
hash codes and modality-specific hash functions. Nonetheless, most existing CMR
works are subjected to some restrictions: 1) It is assumed that data of
different modalities are fully paired, which is impractical in real
applications due to sample missing and false data alignment, and 2) binary
regression targets including the label matrix and binary codes are too rigid to
effectively learn semantic-preserving hash codes and hash functions. To address
these problems, this paper proposes an Adaptive Marginalized Semantic Hashing
(AMSH) method which not only enhances the discrimination of latent
representations and hash codes by adaptive margins, but also can be used for
both paired and unpaired CMR. As a two-step method, in the first step, AMSH
generates semantic-aware modality-specific latent representations with
adaptively marginalized labels, which enlarges the distances between different
classes, and exploits the labels to preserve the inter-modal and intra-modal
semantic similarities into latent representations and hash codes. In the second
step, adaptive margin matrices are embedded into the hash codes, and enlarge
the gaps between positive and negative bits, which improves the discrimination
and robustness of hash functions. On this basis, AMSH generates
similarity-preserving hash codes and robust hash functions without strict
one-to-one data correspondence requirement. Experiments are conducted on
several benchmark datasets to demonstrate the superiority and flexibility of
AMSH over some state-of-the-art CMR methods.
