  Image captioning has emerged as an interesting research field in recent years
due to its broad application scenarios. The traditional paradigm of image
captioning relies on paired image-caption datasets to train the model in a
supervised manner. However, creating such paired datasets for every target
language is prohibitively expensive, which hinders the extensibility of
captioning technology and deprives a large part of the world population of its
benefit. In this work, we present a novel unpaired cross-lingual method to
generate image captions without relying on any caption corpus in the source or
the target language. Specifically, our method consists of two phases: (i) a
cross-lingual auto-encoding process, which utilizing a sentence parallel
(bitext) corpus to learn the mapping from the source to the target language in
the scene graph encoding space and decode sentences in the target language, and
(ii) a cross-modal unsupervised feature mapping, which seeks to map the encoded
scene graph features from image modality to language modality. We verify the
effectiveness of our proposed method on the Chinese image caption generation
task. The comparisons against several existing methods demonstrate the
effectiveness of our approach.
