  This article presents a benchmark study of symbolic piano music
classification using the masked language modelling approach of the
Bidirectional Encoder Representations from Transformers (BERT). Specifically,
we consider two types of MIDI data: MIDI scores, which are musical scores
rendered directly into MIDI with no dynamics and precisely aligned with the
metrical grid notated by its composer and MIDI performances, which are MIDI
encodings of human performances of musical scoresheets. With five public-domain
datasets of single-track piano MIDI files, we pre-train two 12-layer
Transformer models using the BERT approach, one for MIDI scores and the other
for MIDI performances, and fine-tune them for four downstream classification
tasks. These include two note-level classification tasks (melody extraction and
velocity prediction) and two sequence-level classification tasks (style
classification and emotion classification). Our evaluation shows that the BERT
approach leads to higher classification accuracy than recurrent neural network
(RNN)-based baselines.
