  This paper extends the task of probing sentence representations for
linguistic insight in a multilingual domain. In doing so, we make two
contributions: first, we provide datasets for multilingual probing, derived
from Wikipedia, in five languages, viz. English, French, German, Spanish and
Russian. Second, we evaluate six sentence encoders for each language, each
trained by mapping sentence representations to English sentence
representations, using sentences in a parallel corpus. We discover that
cross-lingually mapped representations are often better at retaining certain
linguistic information than representations derived from English encoders
trained on natural language inference (NLI) as a downstream task.
