  Song embeddings are a key component of most music recommendation engines. In
this work, we study the hyper-parameter optimization of behavioral song
embeddings based on Word2Vec on a selection of downstream tasks, namely
next-song recommendation, false neighbor rejection, and artist and genre
clustering. We present new optimization objectives and metrics to monitor the
effects of hyper-parameter optimization. We show that single-objective
optimization can cause side effects on the non optimized metrics and propose a
simple multi-objective optimization to mitigate these effects. We find that
next-song recommendation quality of Word2Vec is anti-correlated with song
popularity, and we show how song embedding optimization can balance performance
across different popularity levels. We then show potential positive downstream
effects on the task of play prediction. Finally, we provide useful insights on
the effects of training dataset scale by testing hyper-parameter optimization
on an industry-scale dataset.
