  Paraphrase generation is a long-standing task in natural language processing
(NLP). Supervised paraphrase generation models, which rely on human-annotated
paraphrase pairs, are cost-inefficient and hard to scale up. On the other hand,
automatically annotated paraphrase pairs (e.g., by machine back-translation),
usually suffer from the lack of syntactic diversity -- the generated paraphrase
sentences are very similar to the source sentences in terms of syntax. In this
work, we present ParaAMR, a large-scale syntactically diverse paraphrase
dataset created by abstract meaning representation back-translation. Our
quantitative analysis, qualitative examples, and human evaluation demonstrate
that the paraphrases of ParaAMR are syntactically more diverse compared to
existing large-scale paraphrase datasets while preserving good semantic
similarity. In addition, we show that ParaAMR can be used to improve on three
NLP tasks: learning sentence embeddings, syntactically controlled paraphrase
generation, and data augmentation for few-shot learning. Our results thus
showcase the potential of ParaAMR for improving various NLP applications.
