  Story visualization aims to generate a sequence of images to narrate each
sentence in a multi-sentence story, where the images should be realistic and
keep global consistency across dynamic scenes and characters. Current works
face the problem of semantic misalignment because of their fixed architecture
and diversity of input modalities. To address this problem, we explore the
semantic alignment between text and image representations by learning to match
their semantic levels in the GAN-based generative model. More specifically, we
introduce dynamic interactions according to learning to dynamically explore
various semantic depths and fuse the different-modal information at a matched
semantic level, which thus relieves the text-image semantic misalignment
problem. Extensive experiments on different datasets demonstrate the
improvements of our approach, neither using segmentation masks nor auxiliary
captioning networks, on image quality and story consistency, compared with
state-of-the-art methods.
