  In recent years, the automatic generation of classical Chinese poetry has
made great progress. Besides focusing on improving the quality of the generated
poetry, there is a new topic about generating poetry from an image. However,
the existing methods for this topic still have the problem of topic drift and
semantic inconsistency, and the image-poem pairs dataset is hard to be built
when training these models. In this paper, we extract and integrate the
Concrete and Abstract information from images to address those issues. We
proposed an infilling-based Chinese poetry generation model which can infill
the Concrete keywords into each line of poems in an explicit way, and an
abstract information embedding to integrate the Abstract information into
generated poems. In addition, we use non-parallel data during training and
construct separate image datasets and poem datasets to train the different
components in our framework. Both automatic and human evaluation results show
that our approach can generate poems which have better consistency with images
without losing the quality.
