  We present an attention-based model for end-to-end handwriting recognition.
Our system does not require any segmentation of the input paragraph. The model
is inspired by the differentiable attention models presented recently for
speech recognition, image captioning or translation. The main difference is the
covert and overt attention, implemented as a multi-dimensional LSTM network.
Our principal contribution towards handwriting recognition lies in the
automatic transcription without a prior segmentation into lines, which was
crucial in previous approaches. To the best of our knowledge this is the first
successful attempt of end-to-end multi-line handwriting recognition. We carried
out experiments on the well-known IAM Database. The results are encouraging and
bring hope to perform full paragraph transcription in the near future.
