  Pretrained language models (PLMs) have been shown to accumulate factual
knowledge during pretrainingng (Petroni et al., 2019). Recent works probe PLMs
for the extent of this knowledge through prompts either in discrete or
continuous forms. However, these methods do not consider symmetry of the task:
object prediction and subject prediction. In this work, we propose Symmetrical
Prompt Enhancement (SPE), a continuous prompt-based method for factual probing
in PLMs that leverages the symmetry of the task by constructing symmetrical
prompts for subject and object prediction. Our results on a popular factual
probing dataset, LAMA, show significant improvement of SPE over previous
probing methods.
