  This paper provides an analysis of character-level machine translation models
used in pivot-based translation when applied to sparse and noisy datasets, such
as crowdsourced movie subtitles. In our experiments, we find that such
character-level models cut the number of untranslated words by over 40% and are
especially competitive (improvements of 2-3 BLEU points) in the case of limited
training data. We explore the impact of character alignment, phrase table
filtering, bitext size and the choice of pivot language on translation quality.
We further compare cascaded translation models to the use of synthetic training
data via multiple pivots, and we find that the latter works significantly
better. Finally, we demonstrate that neither word-nor character-BLEU correlate
perfectly with human judgments, due to BLEU's sensitivity to length.
