  Autoregressive (AR) encoder-decoder neural networks have proved successful in
many NLP problems, including Semantic Parsing -- a task that translates natural
language to machine-readable parse trees. However, the sequential prediction
process of AR models can be slow. To accelerate AR for semantic parsing, we
introduce a new technique called TreePiece that tokenizes a parse tree into
subtrees and generates one subtree per decoding step. On TopV2 benchmark,
TreePiece shows 4.6 times faster decoding speed than standard AR, and
comparable speed but significantly higher accuracy compared to
Non-Autoregressive (NAR).
