  This paper analyzes some speed and performance improvement methods of
Transformer architecture in recent years, mainly its application in dedicated
model training. The dedicated model studied here refers to the open domain
persona-aware dialogue generation model, and the dataset is multi turn short
dialogue, The total length of a single input sequence is no more than 105
tokens. Therefore, many improvements in the architecture and attention
mechanism of transformer architecture for long sequence processing are not
discussed in this paper. The source code of the experiments has been open
sourced: https://github.com/ghosthamlet/persona
