  Matrix factorization has been recently utilized for the task of multi-modal
hashing for cross-modality visual search, where basis functions are learned to
map data from different modalities to the same Hamming embedding. In this
paper, we propose a novel cross-modality hashing algorithm termed Supervised
Matrix Factorization Hashing (SMFH) which tackles the multi-modal hashing
problem with a collective non-matrix factorization across the different
modalities. In particular, SMFH employs a well-designed binary code learning
algorithm to preserve the similarities among multi-modal original features
through a graph regularization. At the same time, semantic labels, when
available, are incorporated into the learning procedure. We conjecture that all
these would facilitate to preserve the most relevant information during the
binary quantization process, and hence improve the retrieval accuracy. We
demonstrate the superior performance of SMFH on three cross-modality visual
search benchmarks, i.e., the PASCAL-Sentence, Wiki, and NUS-WIDE, with
quantitative comparison to various state-of-the-art methods
