  While deep learning technologies are now capable of generating realistic
images confusing humans, the research efforts are turning to the synthesis of
images for more concrete and application-specific purposes. Facial image
generation based on vocal characteristics from speech is one of such important
yet challenging tasks. It is the key enabler to influential use cases of image
generation, especially for business in public security and entertainment.
Existing solutions to the problem of speech2face renders limited image quality
and fails to preserve facial similarity due to the lack of quality dataset for
training and appropriate integration of vocal features. In this paper, we
investigate these key technical challenges and propose Speech Fusion to Face,
or SF2F in short, attempting to address the issue of facial image quality and
the poor connection between vocal feature domain and modern image generation
models. By adopting new strategies on data model and training, we demonstrate
dramatic performance boost over state-of-the-art solution, by doubling the
recall of individual identity, and lifting the quality score from 15 to 19
based on the mutual information score with VGGFace classifier.
