  Amid the rising intersection of generative AI and human artistic processes,
this study probes the critical yet less-explored terrain of alignment in
human-centric automatic song composition. We propose a novel task of Colloquial
Description-to-Song Generation, which focuses on aligning the generated content
with colloquial human expressions. This task is aimed at bridging the gap
between colloquial language understanding and auditory expression within an AI
model, with the ultimate goal of creating songs that accurately satisfy human
auditory expectations and structurally align with musical norms. Current
datasets are limited due to their narrow descriptive scope, semantic gaps and
inaccuracies. To overcome data scarcity in this domain, we present the Caichong
Music Dataset (CaiMD). CaiMD is manually annotated by both professional
musicians and amateurs, offering diverse perspectives and a comprehensive
understanding of colloquial descriptions. Unlike existing datasets pre-set with
expert annotations or auto-generated ones with inherent biases, CaiMD caters
more sufficiently to our purpose of aligning AI-generated music with widespread
user-desired results. Moreover, we propose an innovative single-stage framework
called MuDiT/MuSiT for enabling effective human-machine alignment in song
creation. This framework not only achieves cross-modal comprehension between
colloquial language and auditory music perceptions but also ensures generated
songs align with user-desired results. MuDiT/MuSiT employs one DiT/SiT model
for end-to-end generation of musical components like melody, harmony, rhythm,
vocals, and instrumentation. The approach ensures harmonious sonic cohesiveness
amongst all generated musical components, facilitating better resonance with
human auditory expectations.
