  Generative voice technologies are rapidly evolving, offering opportunities
for more personalized and inclusive experiences. Traditional one-shot voice
conversion (VC) requires a target recording during inference, limiting ease of
usage in generating desired voice timbres. Text-guided generation offers an
intuitive solution to convert voices to desired "DreamVoices" according to the
users' needs. Our paper presents two major contributions to VC technology: (1)
DreamVoiceDB, a robust dataset of voice timbre annotations for 900 speakers
from VCTK and LibriTTS. (2) Two text-guided VC methods: DreamVC, an end-to-end
diffusion-based text-guided VC model; and DreamVG, a versatile text-to-voice
generation plugin that can be combined with any one-shot VC models. The
experimental results demonstrate that our proposed methods trained on the
DreamVoiceDB dataset generate voice timbres accurately aligned with the text
prompt and achieve high-quality VC.
