  Over the past few decades, large archives of paper-based documents such as
books and newspapers have been digitized using Optical Character Recognition.
This technology is error-prone, especially for historical documents. To correct
OCR errors, post-processing algorithms have been proposed based on natural
language analysis and machine learning techniques such as neural networks.
Neural network's disadvantage is the vast amount of manually labeled data
required for training, which is often unavailable. This paper proposes an
innovative method for training a light-weight neural network for Hebrew OCR
post-correction using significantly less manually created data. The main
research goal is to develop a method for automatically generating language and
task-specific training data to improve the neural network results for OCR
post-correction, and to investigate which type of dataset is the most effective
for OCR post-correction of historical documents. To this end, a series of
experiments using several datasets was conducted. The evaluation corpus was
based on Hebrew newspapers from the JPress project. An analysis of historical
OCRed newspapers was done to learn common language and corpus-specific OCR
errors. We found that training the network using the proposed method is more
effective than using randomly generated errors. The results also show that the
performance of the neural network for OCR post-correction strongly depends on
the genre and area of the training data. Moreover, neural networks that were
trained with the proposed method outperform other state-of-the-art neural
networks for OCR post-correction and complex spellcheckers. These results may
have practical implications for many digital humanities projects.
