  Progress in sentence simplification has been hindered by a lack of labeled
parallel simplification data, particularly in languages other than English. We
introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that
does not require labeled simplification data. MUSS uses a novel approach to
sentence simplification that trains strong models using sentence-level
paraphrase data instead of proper simplification data. These models leverage
unsupervised pretraining and controllable generation mechanisms to flexibly
adjust attributes such as length and lexical complexity at inference time. We
further present a method to mine such paraphrase data in any language from
Common Crawl using semantic sentence embeddings, thus removing the need for
labeled data. We evaluate our approach on English, French, and Spanish
simplification benchmarks and closely match or outperform the previous best
supervised results, despite not using any labeled simplification data. We push
the state of the art further by incorporating labeled simplification data.
