  We present a novel neural model for modern poetry generation in French. The
model consists of two pretrained neural models that are fine-tuned for the poem
generation task. The encoder of the model is a RoBERTa based one while the
decoder is based on GPT-2. This way the model can benefit from the superior
natural language understanding performance of RoBERTa and the good natural
language generation performance of GPT-2. Our evaluation shows that the model
can create French poetry successfully. On a 5 point scale, the lowest score of
3.57 was given by human judges to typicality and emotionality of the output
poetry while the best score of 3.79 was given to understandability.
