  Singing voice conversion (SVC) aims to convert a singer's voice in a given
music piece to another singer while keeping the original content. We propose an
end-to-end feature disentanglement-based model, which we named SaMoye, to
enable zero-shot many-to-many singing voice conversion. SaMoye disentangles the
features of the singing voice into content features, timbre features, and pitch
features respectively. The content features are enhanced using a GPT-based
model to perform cross-prediction with the phoneme of the lyrics. SaMoye can
generate the music with converted voice by replacing the timbre features with
the target singer. We also establish an unparalleled large-scale dataset to
guarantee zero-shot performance. The dataset consists of 1500k pure singing
vocal clips containing at least 10,000 singers.
