  Traditionally, character-level transduction problems have been solved with
finite-state models designed to encode structural and linguistic knowledge of
the underlying process, whereas recent approaches rely on the power and
flexibility of sequence-to-sequence models with attention. Focusing on the less
explored unsupervised learning scenario, we compare the two model classes side
by side and find that they tend to make different types of errors even when
achieving comparable performance. We analyze the distributions of different
error classes using two unsupervised tasks as testbeds: converting informally
romanized text into the native script of its language (for Russian, Arabic, and
Kannada) and translating between a pair of closely related languages (Serbian
and Bosnian). Finally, we investigate how combining finite-state and
sequence-to-sequence models at decoding time affects the output quantitatively
and qualitatively.
