  Score following is the process of tracking a musical performance (audio) with
respect to a known symbolic representation (a score). We start this paper by
formulating score following as a multimodal Markov Decision Process, the
mathematical foundation for sequential decision making. Given this formal
definition, we address the score following task with state-of-the-art deep
reinforcement learning (RL) algorithms such as synchronous advantage actor
critic (A2C). In particular, we design multimodal RL agents that simultaneously
learn to listen to music, read the scores from images of sheet music, and
follow the audio along in the sheet, in an end-to-end fashion. All this
behavior is learned entirely from scratch, based on a weak and potentially
delayed reward signal that indicates to the agent how close it is to the
correct position in the score. Besides discussing the theoretical advantages of
this learning paradigm, we show in experiments that it is in fact superior
compared to previously proposed methods for score following in raw sheet music
images.
