  Representing symbolic music with compound tokens, where each token consists
of several different sub-tokens representing a distinct musical feature or
attribute, offers the advantage of reducing sequence length. While previous
research has validated the efficacy of compound tokens in music sequence
modeling, predicting all sub-tokens simultaneously can lead to suboptimal
results as it may not fully capture the interdependencies between them. We
introduce the Nested Music Transformer (NMT), an architecture tailored for
decoding compound tokens autoregressively, similar to processing flattened
tokens, but with low memory usage. The NMT consists of two transformers: the
main decoder that models a sequence of compound tokens and the sub-decoder for
modeling sub-tokens of each compound token. The experiment results showed that
applying the NMT to compound tokens can enhance the performance in terms of
better perplexity in processing various symbolic music datasets and discrete
audio tokens from the MAESTRO dataset.
