  We present JASCO, a temporally controlled text-to-music generation model
utilizing both symbolic and audio-based conditions. JASCO can generate
high-quality music samples conditioned on global text descriptions along with
fine-grained local controls. JASCO is based on the Flow Matching modeling
paradigm together with a novel conditioning method. This allows music
generation controlled both locally (e.g., chords) and globally (text
description). Specifically, we apply information bottleneck layers in
conjunction with temporal blurring to extract relevant information with respect
to specific controls. This allows the incorporation of both symbolic and
audio-based conditions in the same text-to-music model. We experiment with
various symbolic control signals (e.g., chords, melody), as well as with audio
representations (e.g., separated drum tracks, full-mix). We evaluate JASCO
considering both generation quality and condition adherence, using both
objective metrics and human studies. Results suggest that JASCO is comparable
to the evaluated baselines considering generation quality while allowing
significantly better and more versatile controls over the generated music.
Samples are available on our demo page
https://pages.cs.huji.ac.il/adiyoss-lab/JASCO.
