  The heterogeneity gap problem is the main challenge in cross-modal retrieval.
Because cross-modal data (e.g. audiovisual) have different distributions and
representations that cannot be directly compared. To bridge the gap between
audiovisual modalities, we learn a common subspace for them by utilizing the
intrinsic correlation in the natural synchronization of audio-visual data with
the aid of annotated labels. TNN-CCCA is the best audio-visual cross-modal
retrieval (AV-CMR) model so far, but the model training is sensitive to hard
negative samples when learning common subspace by applying triplet loss to
predict the relative distance between inputs. In this paper, to reduce the
interference of hard negative samples in representation learning, we propose a
new AV-CMR model to optimize semantic features by directly predicting labels
and then measuring the intrinsic correlation between audio-visual data using
complete cross-triple loss. In particular, our model projects audio-visual
features into label space by minimizing the distance between predicted label
features after feature projection and ground label representations. Moreover,
we adopt complete cross-triplet loss to optimize the predicted label features
by leveraging the relationship between all possible similarity and
dissimilarity semantic information across modalities. The extensive
experimental results on two audio-visual double-checked datasets have shown an
improvement of approximately 2.1% in terms of average MAP over the current
state-of-the-art method TNN-CCCA for the AV-CMR task, which indicates the
effectiveness of our proposed model.
