  Patients take care of what their teeth will be like after the orthodontics.
Orthodontists usually describe the expectation movement based on the original
smile images, which is unconvincing. The growth of deep-learning generative
models change this situation. It can visualize the outcome of orthodontic
treatment and help patients foresee their future teeth and facial appearance.
While previous studies mainly focus on 2D or 3D virtual treatment outcome (VTO)
at a profile level, the problem of simulating treatment outcome at a frontal
facial image is poorly explored. In this paper, we build an efficient and
accurate system for simulating virtual teeth alignment effects in a frontal
facial image. Our system takes a frontal face image of a patient with visible
malpositioned teeth and the patient's 3D scanned teeth model as input, and
progressively generates the visual results of the patient's teeth given the
specific orthodontics planning steps from the doctor (i.e., the specification
of translations and rotations of individual tooth). We design a multi-modal
encoder-decoder based generative model to synthesize identity-preserving
frontal facial images with aligned teeth. In addition, the original image color
information is used to optimize the orthodontic outcomes, making the results
more natural. We conduct extensive qualitative and clinical experiments and
also a pilot study to validate our method.
