  Tag-based music retrieval is crucial to browse large-scale music libraries
efficiently. Hence, automatic music tagging has been actively explored, mostly
as a classification task, which has an inherent limitation: a fixed vocabulary.
On the other hand, metric learning enables flexible vocabularies by using
pretrained word embeddings as side information. Also, metric learning has
already proven its suitability for cross-modal retrieval tasks in other domains
(e.g., text-to-image) by jointly learning a multimodal embedding space. In this
paper, we investigate three ideas to successfully introduce multimodal metric
learning for tag-based music retrieval: elaborate triplet sampling, acoustic
and cultural music information, and domain-specific word embeddings. Our
experimental results show that the proposed ideas enhance the retrieval system
quantitatively, and qualitatively. Furthermore, we release the MSD500, a subset
of the Million Song Dataset (MSD) containing 500 cleaned tags, 7 manually
annotated tag categories, and user taste profiles.
