  Text-to-speech(TTS) has undergone remarkable improvements in performance,
particularly with the advent of Denoising Diffusion Probabilistic Models
(DDPMs). However, the perceived quality of audio depends not solely on its
content, pitch, rhythm, and energy, but also on the physical environment. In
this work, we propose ViT-TTS, the first visual TTS model with scalable
diffusion transformers. ViT-TTS complement the phoneme sequence with the visual
information to generate high-perceived audio, opening up new avenues for
practical applications of AR and VR to allow a more immersive and realistic
audio experience. To mitigate the data scarcity in learning visual acoustic
information, we 1) introduce a self-supervised learning framework to enhance
both the visual-text encoder and denoiser decoder; 2) leverage the diffusion
transformer scalable in terms of parameters and capacity to learn visual scene
information. Experimental results demonstrate that ViT-TTS achieves new
state-of-the-art results, outperforming cascaded systems and other baselines
regardless of the visibility of the scene. With low-resource data (1h, 2h, 5h),
ViT-TTS achieves comparative results with rich-resource
baselines.~\footnote{Audio samples are available at
\url{https://ViT-TTS.github.io/.}}
