  Binarization of document images is an important pre-processing step in the
field of document analysis. Traditional image binarization techniques usually
rely on histograms or local statistics to identify a valid threshold to
differentiate between different aspects of the image. Deep learning techniques
are able to generate binarized versions of the images by learning
context-dependent features that are less error-prone to degradation typically
occurring in document images. In recent years, many deep learning-based methods
have been developed for document binarization. But which one to choose? There
have been no studies that compare these methods rigorously. Therefore, this
work focuses on the evaluation of different deep learning-based methods under
the same evaluation protocol. We evaluate them on different Document Image
Binarization Contest (DIBCO) datasets and obtain very heterogeneous results. We
show that the DE-GAN model was able to perform better compared to other models
when evaluated on the DIBCO2013 dataset while DP-LinkNet performed best on the
DIBCO2017 dataset. The 2-StageGAN performed best on the DIBCO2018 dataset while
SauvolaNet outperformed the others on the DIBCO2019 challenge. Finally, we make
the code, all models and evaluation publicly available
(https://github.com/RichSu95/Document_Binarization_Collection) to ensure
reproducibility and simplify future binarization evaluations.
