  Masked language models (MLM) do not explicitly define a distribution over
language, i.e., they are not language models per se. However, recent work has
implicitly treated them as such for the purposes of generation and scoring.
This paper studies methods for deriving explicit joint distributions from MLMs,
focusing on distributions over two tokens, which makes it possible to calculate
exact distributional properties. We find that an approach based on identifying
joints whose conditionals are closest to those of the MLM works well and
outperforms existing Markov random field-based approaches. We further find that
this derived model's conditionals can even occasionally outperform the original
MLM's conditionals.
