  We show that state-of-the-art self-supervised language models can be readily
used to extract relations from a corpus without the need to train a fine-tuned
extractive head. We introduce RE-Flex, a simple framework that performs
constrained cloze completion over pretrained language models to perform
unsupervised relation extraction. RE-Flex uses contextual matching to ensure
that language model predictions matches supporting evidence from the input
corpus that is relevant to a target relation. We perform an extensive
experimental study over multiple relation extraction benchmarks and demonstrate
that RE-Flex outperforms competing unsupervised relation extraction methods
based on pretrained language models by up to 27.8 $F_1$ points compared to the
next-best method. Our results show that constrained inference queries against a
language model can enable accurate unsupervised relation extraction.
