  We present a simple yet effective method for generating high quality
classical Chinese poetry with Generative Pre-trained Language Model (GPT). The
method adopts a simple GPT model, without using any human crafted rules or
features, or designing any additional neural components. While the proposed
model learns to generate various forms of classical Chinese poems, including
Jueju, L\"{u}shi, various Cipai and Couples, the generated poems are of very
high quality. We also propose and implement a method to fine-tune the model to
generate acrostic poetry. To the best of our knowledge, this is the first to
employ GPT in developing a poetry generation system. We have released an online
mini demonstration program on Wechat to show the generation capability of the
proposed method for classical Chinese poetry.
