  Pre-trained Language Models (PLMs) are trained on vast unlabeled data, rich
in world knowledge. This fact has sparked the interest of the community in
quantifying the amount of factual knowledge present in PLMs, as this explains
their performance on downstream tasks, and potentially justifies their use as
knowledge bases. In this work, we survey methods and datasets that are used to
probe PLMs for factual knowledge. Our contributions are: (1) We propose a
categorization scheme for factual probing methods that is based on how their
inputs, outputs and the probed PLMs are adapted; (2) We provide an overview of
the datasets used for factual probing; (3) We synthesize insights about
knowledge retention and prompt optimization in PLMs, analyze obstacles to
adopting PLMs as knowledge bases and outline directions for future work.
