  The ACM Recommender Systems Challenge 2018 focused on the task of automatic
music playlist continuation, which is a form of the more general task of
sequential recommendation. Given a playlist of arbitrary length with some
additional meta-data, the task was to recommend up to 500 tracks that fit the
target characteristics of the original playlist. For the RecSys Challenge,
Spotify released a dataset of one million user-generated playlists.
Participants could compete in two tracks, i.e., main and creative tracks.
Participants in the main track were only allowed to use the provided training
set, however, in the creative track, the use of external public sources was
permitted. In total, 113 teams submitted 1,228 runs to the main track; 33 teams
submitted 239 runs to the creative track. The highest performing team in the
main track achieved an R-precision of 0.2241, an NDCG of 0.3946, and an average
number of recommended songs clicks of 1.784. In the creative track, an
R-precision of 0.2233, an NDCG of 0.3939, and a click rate of 1.785 was
obtained by the best team. This article provides an overview of the challenge,
including motivation, task definition, dataset description, and evaluation. We
further report and analyze the results obtained by the top performing teams in
each track and explore the approaches taken by the winners. We finally
summarize our key findings, discuss generalizability of approaches and results
to domains other than music, and list the open avenues and possible future
directions in the area of automatic playlist continuation.
