  We present an evaluation of text simplification (TS) in Spanish for a
production system, by means of two corpora focused in both complex-sentence and
complex-word identification. We compare the most prevalent Spanish-specific
readability scores with neural networks, and show that the latter are
consistently better at predicting user preferences regarding TS. As part of our
analysis, we find that multilingual models underperform against equivalent
Spanish-only models on the same task, yet all models focus too often on
spurious statistical features, such as sentence length. We release the corpora
in our evaluation to the broader community with the hopes of pushing forward
the state-of-the-art in Spanish natural language processing.
