  We present a challenge set for French --> English machine translation based
on the approach introduced in Isabelle, Cherry and Foster (EMNLP 2017). Such
challenge sets are made up of sentences that are expected to be relatively
difficult for machines to translate correctly because their most
straightforward translations tend to be linguistically divergent. We present
here a set of 506 manually constructed French sentences, 307 of which are
targeted to the same kinds of structural divergences as in the paper mentioned
above. The remaining 199 sentences are designed to test the ability of the
systems to correctly translate difficult grammatical words such as
prepositions. We report on the results of using this challenge set for testing
two different systems, namely Google Translate and DEEPL, each on two different
dates (October 2017 and January 2018). All the resulting data are made publicly
available.
