  This report explores the challenge of enhancing expressiveness control in
Text-to-Speech (TTS) models by augmenting a frozen pretrained model with a
Diffusion Model that is conditioned on joint semantic audio/text embeddings.
The paper identifies the challenges encountered when working with a VAE-based
TTS model and evaluates different image-to-image methods for altering latent
speech features. Our results offer valuable insights into the complexities of
adding expressiveness control to TTS systems and open avenues for future
research in this direction.
