  Pretrained language models are generally acknowledged to be able to encode
syntax [Tenney et al., 2019, Jawahar et al., 2019, Hewitt and Manning, 2019].
In this article, we propose UPOA, an Unsupervised constituent Parsing model
that calculates an Out Association score solely based on the self-attention
weight matrix learned in a pretrained language model as the syntactic distance
for span segmentation. We further propose an enhanced version, UPIO, which
exploits both inside association and outside association scores for estimating
the likelihood of a span. Experiments with UPOA and UPIO disclose that the
linear projection matrices for the query and key in the self-attention
mechanism play an important role in parsing. We therefore extend the
unsupervised models to few-shot parsing models (FPOA, FPIO) that use a few
annotated trees to learn better linear projection matrices for parsing.
Experiments on the Penn Treebank demonstrate that our unsupervised parsing
model UPIO achieves results comparable to the state of the art on short
sentences (length <= 10). Our few-shot parsing model FPIO trained with only 20
annotated trees outperforms a previous few-shot parsing method trained with 50
annotated trees. Experiments on cross-lingual parsing show that both
unsupervised and few-shot parsing methods are better than previous methods on
most languages of SPMRL [Seddah et al., 2013].
