  Literary translation is a culturally significant task, but it is bottlenecked
by the small number of qualified literary translators relative to the many
untranslated works published around the world. Machine translation (MT) holds
potential to complement the work of human translators by improving both
training procedures and their overall efficiency. Literary translation is less
constrained than more traditional MT settings since translators must balance
meaning equivalence, readability, and critical interpretability in the target
language. This property, along with the complex discourse-level context present
in literary texts, also makes literary MT more challenging to computationally
model and evaluate. To explore this task, we collect a dataset (Par3) of
non-English language novels in the public domain, each aligned at the paragraph
level to both human and automatic English translations. Using Par3, we discover
that expert literary translators prefer reference human translations over
machine-translated paragraphs at a rate of 84%, while state-of-the-art
automatic MT metrics do not correlate with those preferences. The experts note
that MT outputs contain not only mistranslations, but also discourse-disrupting
errors and stylistic inconsistencies. To address these problems, we train a
post-editing model whose output is preferred over normal MT output at a rate of
69% by experts. We publicly release Par3 at
https://github.com/katherinethai/par3/ to spur future research into literary
MT.
