  We show that Bayes' rule provides an effective mechanism for creating
document translation models that can be learned from only parallel sentences
and monolingual documents---a compelling benefit as parallel documents are not
always available. In our formulation, the posterior probability of a candidate
translation is the product of the unconditional (prior) probability of the
candidate output document and the "reverse translation probability" of
translating the candidate output back into the source language. Our proposed
model uses a powerful autoregressive language model as the prior on target
language documents, but it assumes that each sentence is translated
independently from the target to the source language. Crucially, at test time,
when a source document is observed, the document language model prior induces
dependencies between the translations of the source sentences in the posterior.
The model's independence assumption not only enables efficient use of available
data, but it additionally admits a practical left-to-right beam-search
algorithm for carrying out inference. Experiments show that our model benefits
from using cross-sentence context in the language model, and it outperforms
existing document translation approaches.
