  Sequence-to-sequence models have recently become very popular for tackling
handwritten word recognition problems. However, how to effectively integrate an
external language model into such recognizer is still a challenging problem.
The main challenge faced when training a language model is to deal with the
language model corpus which is usually different to the one used for training
the handwritten word recognition system. Thus, the bias between both word
corpora leads to incorrectness on the transcriptions, providing similar or even
worse performances on the recognition task. In this work, we introduce
Candidate Fusion, a novel way to integrate an external language model to a
sequence-to-sequence architecture. Moreover, it provides suggestions from an
external language knowledge, as a new input to the sequence-to-sequence
recognizer. Hence, Candidate Fusion provides two improvements. On the one hand,
the sequence-to-sequence recognizer has the flexibility not only to combine the
information from itself and the language model, but also to choose the
importance of the information provided by the language model. On the other
hand, the external language model has the ability to adapt itself to the
training corpus and even learn the most commonly errors produced from the
recognizer. Finally, by conducting comprehensive experiments, the Candidate
Fusion proves to outperform the state-of-the-art language models for
handwritten word recognition tasks.
