  Offline handwriting recognition (HWR) has improved significantly with the
advent of deep learning architectures in recent years. Nevertheless, it remains
a challenging problem and practical applications often rely on post-processing
techniques for restricting the predicted words via lexicons or language models.
Despite their enhanced performance, such systems are less usable in contexts
where out-of-vocabulary words are anticipated, e.g. for detecting misspelled
words in school assessments. To that end, we introduce the task of comparing a
handwriting image to text. To solve the problem, we propose an unrestricted
binary classifier, consisting of a HWR feature extractor and a multimodal
classification head which convolves the feature extractor output with the
vector representation of the input text. Our model's classification head is
trained entirely on synthetic data created using a state-of-the-art generative
adversarial network. We demonstrate that, while maintaining high recall, the
classifier can be calibrated to achieve an average precision increase of 19.5%
compared to addressing the task by directly using state-of-the-art HWR models.
Such massive performance gains can lead to significant productivity increases
in applications utilizing human-in-the-loop automation.
