  In the literature, the research on abstract meaning representation (AMR)
parsing is much restricted by the size of human-curated dataset which is
critical to build an AMR parser with good performance. To alleviate such data
size restriction, pre-trained models have been drawing more and more attention
in AMR parsing. However, previous pre-trained models, like BERT, are
implemented for general purpose which may not work as expected for the specific
task of AMR parsing. In this paper, we focus on sequence-to-sequence (seq2seq)
AMR parsing and propose a seq2seq pre-training approach to build pre-trained
models in both single and joint way on three relevant tasks, i.e., machine
translation, syntactic parsing, and AMR parsing itself. Moreover, we extend the
vanilla fine-tuning method to a multi-task learning fine-tuning method that
optimizes for the performance of AMR parsing while endeavors to preserve the
response of pre-trained models. Extensive experimental results on two English
benchmark datasets show that both the single and joint pre-trained models
significantly improve the performance (e.g., from 71.5 to 80.2 on AMR 2.0),
which reaches the state of the art. The result is very encouraging since we
achieve this with seq2seq models rather than complex models. We make our code
and model available at https://github.com/xdqkid/S2S-AMR-Parser.
