  Formal verse poetry imposes strict constraints on the meter and rhyme scheme
of poems. Most prior work on generating this type of poetry uses existing poems
for supervision, which are difficult to obtain for most languages and poetic
forms. In this work, we propose an unsupervised approach to generate poems
following any given meter and rhyme scheme, without requiring any poetic text
for training. Our method works by splitting a regular, non-poetic corpus into
phrases, prepending control codes that describe the length and end rhyme of
each phrase, and training a transformer language model in the augmented corpus.
During inference, we build control codes for the desired meter and rhyme
scheme, and condition our language model on them to generate formal verse
poetry. Experiments in Spanish and Basque show that our approach is able to
generate valid poems, which are often comparable in quality to those written by
humans.
