  We propose a new unsupervised lexical simplification method that uses only
monolingual data and pre-trained language models. Given a target word and its
context, our method generates substitutes based on the target context and also
additional contexts sampled from monolingual data. We conduct experiments in
English, Portuguese, and Spanish on the TSAR-2022 shared task, and show that
our model substantially outperforms other unsupervised systems across all
languages. We also establish a new state-of-the-art by ensembling our model
with GPT-3.5. Lastly, we evaluate our model on the SWORDS lexical substitution
data set, achieving a state-of-the-art result.
