  Existing music generation models are mostly language-based, neglecting the
frequency continuity property of notes, resulting in inadequate fitting of rare
or never-used notes and thus reducing the diversity of generated samples. We
argue that the distribution of notes can be modeled by translational invariance
and periodicity, especially using diffusion models to generalize notes by
injecting frequency-domain Gaussian noise. However, due to the low-density
nature of music symbols, estimating the distribution of notes latent in the
high-density solution space poses significant challenges. To address this
problem, we introduce the Music-Diff architecture, which fits a joint
distribution of notes and accompanying semantic information to generate
symbolic music conditionally. We first enhance the fragmentation module for
extracting semantics by using event-based notations and the structural
similarity index, thereby preventing boundary blurring. As a prerequisite for
multivariate perturbation, we introduce a joint pre-training method to
construct the progressions between notes and musical semantics while avoiding
direct modeling of low-density notes. Finally, we recover the perturbed notes
by a multi-branch denoiser that fits multiple noise objectives via Pareto
optimization. Our experiments suggest that in contrast to language models,
joint probability diffusion models perturbing at both note and semantic levels
can provide more sample diversity and compositional regularity. The case study
highlights the rhythmic advantages of our model over language- and DDPMs-based
models by analyzing the hierarchical structure expressed in the self-similarity
metrics.
