  In this work, we propose EASSE-multi, a framework for easier automatic
sentence evaluation for languages other than English. Compared to the original
EASSE framework, EASSE-multi does not focus only on English. It contains
tokenizers and versions of text simplification evaluation metrics which are
suitable for multiple languages. In this paper, we exemplify the usage of
EASSE-multi for German TS, resulting in EASSE-DE. Further, we compare text
simplification results when evaluating with different language or tokenization
settings of the metrics. Based on this, we formulate recommendations on how to
make the evaluation of (German) TS models more transparent and better
comparable. The code of EASSE-multi and its German specialisation (EASSE-DE)
can be found at https://github.com/rstodden/easse-de.
