  We propose the Recursive Non-autoregressive Graph-to-Graph Transformer
architecture (RNGTr) for the iterative refinement of arbitrary graphs through
the recursive application of a non-autoregressive Graph-to-Graph Transformer
and apply it to syntactic dependency parsing. We demonstrate the power and
effectiveness of RNGTr on several dependency corpora, using a refinement model
pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a
non-recursive parser similar to our refinement model. RNGTr can improve the
accuracy of a variety of initial parsers on 13 languages from the Universal
Dependencies Treebanks, English and Chinese Penn Treebanks, and the German
CoNLL2009 corpus, even improving over the new state-of-the-art results achieved
by SynTr, significantly improving the state-of-the-art for all corpora tested.
