  Learning symbolic music representations, especially disentangled
representations with probabilistic interpretations, has been shown to benefit
both music understanding and generation. However, most models are only
applicable to short-term music, while learning long-term music representations
remains a challenging task. We have seen several studies attempting to learn
hierarchical representations directly in an end-to-end manner, but these models
have not been able to achieve the desired results and the training process is
not stable. In this paper, we propose a novel approach to learn long-term
symbolic music representations through contextual constraints. First, we use
contrastive learning to pre-train a long-term representation by constraining
its difference from the short-term representation (extracted by an
off-the-shelf model). Then, we fine-tune the long-term representation by a
hierarchical prediction model such that a good long-term representation (e.g.,
an 8-bar representation) can reconstruct the corresponding short-term ones
(e.g., the 2-bar representations within the 8-bar range). Experiments show that
our method stabilizes the training and the fine-tuning steps. In addition, the
designed contextual constraints benefit both reconstruction and
disentanglement, significantly outperforming the baselines.
