  Subword tokenization has been widely successful in text-based natural
language processing (NLP) tasks with Transformer-based models. As Transformer
models become increasingly popular in symbolic music-related studies, it is
imperative to investigate the efficacy of subword tokenization in the symbolic
music domain. In this paper, we explore subword tokenization techniques, such
as byte-pair encoding (BPE), in symbolic music generation and its impact on the
overall structure of generated songs. Our experiments are based on three types
of MIDI datasets: single track-melody only, multi-track with a single
instrument, and multi-track and multi-instrument. We apply subword tokenization
on post-musical tokenization schemes and find that it enables the generation of
longer songs at the same time and improves the overall structure of the
generated music in terms of objective metrics like structure indicator (SI),
Pitch Class Entropy, etc. We also compare two subword tokenization methods, BPE
and Unigram, and observe that both methods lead to consistent improvements. Our
study suggests that subword tokenization is a promising technique for symbolic
music generation and may have broader implications for music composition,
particularly in cases involving complex data such as multi-track songs.
