A selectional auto-encoder approach for document image binarization
Binarization plays a key role in the automatic information retrieval from document images. This process is usually performed in the first stages of documents analysis systems, and serves as a basis for subsequent steps. Hence it has to be robust in order to allow the full analysis workflow to be successful. Several methods for document image binarization have been proposed so far, most of which are based on hand-crafted image processing strategies. Recently, Convolutional Neural Networks have shown an amazing performance in many disparate duties related to computer vision. In this paper we discuss the use of convolutional auto-encoders devoted to learning an end-to-end map from an input image to its selectional output, in which activations indicate the likelihood of pixels to be either foreground or background. Once trained, documents can therefore be binarized by parsing them through the model and applying a threshold. This approach has proven to outperform existing binarization strategies in a number of document domains.
binarization.tex
\documentclass{elsarticle}

\usepackage{lineno,hyperref}
\modulolinenumbers[5]

\journal{Journal}

\usepackage{subcaption}
\usepackage{amssymb}
\usepackage{amstext}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage[usenames,dvipsnames]{xcolor}
\usepackage{multirow}
\usepackage{dcolumn}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\graphicspath{{./figs/}}
\DeclareGraphicsExtensions{.pdf,.png}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{verbatim}
\usepackage[colorinlistoftodos,textwidth=4cm]{todonotes}
\usepackage[normalem]{ulem}



\usepackage{ctable}
\usepackage{setspace}

% \newcolumntype{.}{D{.}{.}{-1}}
\usepackage{soul}
\newcommand{\javi}[1]{\textcolor{OliveGreen}{#1}}
\newcommand{\jorge}[1]{\textcolor{Sepia}{#1}}



\DeclareMathOperator{\CAE}{SAE}

\DeclareMathOperator{\TP}{TP}
\DeclareMathOperator{\FP}{FP}
\DeclareMathOperator{\TN}{TN}
\DeclareMathOperator{\FN}{FN}


\begin{document}

\begin{frontmatter}

\title{A selectional auto-encoder approach for document image binarization}

\author[add1]{Jorge Calvo-Zaragoza\corref{cor1}}
%\ead{jcalvo@dlsi.ua.es}
\ead{jorge.calvozaragoza@mail.mcgill.ca}
\cortext[cor1]{Corresponding author: 
Tel.: +346-66-837771}

\author[add1]{Antonio-Javier Gallego}
\ead{ajgallego@dlsi.ua.es}

\address[add1]{Schulich School of Music, McGill University, 555 Sherbrooke St W, Montreal, QC H3A 1E3, Canada}

\address[add2]{Department of Software and Computing Systems, University of Alicante, Carretera San Vicente del Raspeig s/n, 03690 Alicante, Spain}

\date{\today}

\begin{abstract}
Binarization plays a key role in the automatic information retrieval from document images. This process is usually performed in the first stages of documents analysis systems, and serves as a basis for subsequent steps. Hence it has to be robust in order to allow the full analysis workflow to be successful. Several methods for document image binarization have been proposed so far, most of which are based on hand-crafted image processing strategies. Recently, Convolutional Neural Networks have shown an amazing performance in many disparate duties related to computer vision. In this paper we discuss the use of convolutional auto-encoders devoted to learning an end-to-end map from an input image to its selectional output, in which activations indicate the likelihood of pixels to be either foreground or background. Once trained, documents can therefore be binarized by parsing them through the model and applying a threshold. This approach has proven to outperform existing binarization strategies in a number of document domains.
\end{abstract}

\begin{keyword}
Binarization \sep Document Analysis \sep Auto-encoders \sep Convolutional Neural Networks 
\end{keyword}

\end{frontmatter}


% ==================
% == INTRODUCCION ==
% ==================
\section{Introduction}
\label{sec:introduction}
Image binarization consists in assigning a binary value to every single pixel of the image. Within the context of document analysis systems, the main objective is to distinguish the \emph{foreground} (meaningful information) from the \emph{background}.

Binarization plays a key role in the workflow of many document analysis and recognition systems \cite{LOULOUDIS20083758,saabni2014text,HE20154036,calvo2015avoiding,GIOTIS2017310}. It not only helps to reduce the complexity of the task but it is also advisable for procedures involving morphological operations, detection of connected components, or histogram analysis, among others. Many methods have been proposed to accomplish this task. However, it is often complex to attain good results because documents may contain several difficulties---such as irregular leveling, blots, bleed-through, and so on---that may cause the process to fail.

In addition to all these obstacles, it is convenient to emphasize that it is very difficult for the same method to work successfully in a number of document styles, since the set of potential domains is very heterogeneous. In order to deal with this situation, we discuss a framework with which to binarize image documents based on machine learning. That is, a ground-truth of examples is used to train a model to perform the binarization task. This allows using the same approach in a wide range of documents, as long as training data is available.

Specifically, we make use of Convolutional Neural Networks (CNN) \cite{zeiler2014visualizing}. These networks involve multi-layer architectures that perform a series of transformations (convolutions) to the input signal. The parameters of these transformations are adjusted through a training process. CNN have drastically improved the state of the art in many tasks such as image, video, and speech recognition~\cite{lecun15deep}. Thus, its use for document binarization is promising. In this case, we consider an image-to-image convolutional architecture, which is trained to transform an input image into its binarized version. 

The idea of this approach is that it allows adapting easily the model to a wider range of domains as long as proper training data is available. Our experiments, therefore, focus on testing the algorithm in different domains, namely Latin text documents, palm leaf scripts, Persian documents, and music scores. We also compare our approach against other classical and state-of-the-art algorithms for binarization, showing that this approach leads to a significant improvement.

The remainder of the paper is structured as follows: related works to document image binarization are introduced in Section~\ref{sec:background}; the novel binarization framework based on convolutional models is described in Section~\ref{sec:method}; the experiments are presented in Section~\ref{sec:experiments}, including model configuration, training strategies, and comparison with existing techniques; and finally, the current work concludes in Section~\ref{sec:conclusions}.

\section{Background} \label{sec:background}
The most straightforward procedure for image binarization is to resort to simple thresholding, in which all pixels under a certain grayscale value are set to $0$, and those above to $1$. This threshold can be fixed by hand, yet algorithms such as Otsu's \cite{otsu1975threshold} automatically estimate a value according to the input image.

However, as the complexity of the document to process increases, this simple procedure usually leads to poor or irregular binarization, and so it is preferable to resort to other kind of approaches.

Instead of computing a single global threshold, different threshold values might be obtained for every pixel by taking into account the features of its local neighborhood (defined by a window centered around the pixel). Niblack's algorithm~\cite{niblack1985introduction} was one of the first binarization processes following this approach. Let $m$ and $s$ be the mean and standard deviation, respectively, of the considered neighboring region, it computes the threshold as $T = m + k \cdot s$, where $k$ is a parameter to tune. 

Most common extensions to this approach are Sauvola's~\cite{sauvola2000adaptive} and that proposed by Wolf et al.~\cite{wolf02text}, which try to boost the accuracy of the binarization by considering more complex equations for the adaptive threshold. The former assumes that foreground pixels are closer to black than background pixels, whereas the latter normalizes contrast and mean gray-level of the considered neighborhood. Recently, Lazzara and Geraud proposed an efficient multi-scale version of Sauvola's method (Sauvola MS) to deal better with documents depicting components of different sizes \cite{lazzara2014efficient}. 

Posterior to these methods, there have been proposed several procedures for document image binarization that add more steps to the pipeline. Gatos method~\cite{gatos06adaptive} is an adaptive procedure that follows several steps, namely a low-pass Wiener filter, estimation of foreground and background regions, and a thresholding. It ultimately applies a post-processing step to improve the quality of foreground regions and preserve stroke connectivity. The method proposed by Su et al.~\cite{su2013robust} uses an adaptive image contrast as a combination of the local image contrast and the local image gradient, which makes it more robust against document degradations. The contrast map is then combined with an edge detector to identify the boundaries of foreground elements. Then, the document is finally binarized by using a local threshold based on the values of those boundaries. Howe's method \cite{howe2011laplacian} considers the Laplacian operator to compute the local likelihood of foreground and background labels, and Canny algorithm to detect discontinuities. It then formulates the binarization task as a global energy minimization problem. The method contains several parameters, which can be automatically tuned following a subsequent work \cite{howe2013document}. Very recently, Howe's method has been combined with a new preprocessing that boosts its performance. It linearly transforms the image onto a spherical surface in which concavities correspond to foreground in the original image. The concavities are estimated with the Hidden Point Removal operator \cite{katz2007direct}, which produces a probability for each pixel of the image to belong to a concavity. This stochastic representation is then used as input for Howe's binarization. This strategy was proposed by Kliger and Tal in the latest Document Image Binarization Contest \cite{PratikakisZBG16}, held within the International Conference on Frontiers in Handwriting Recognition, in which it was ranked first.

All aforementioned methods have been successful in their contexts. However, it is difficult for these strategies to generalize adequately to any type of document. It is, therefore, interesting to resort to binarization procedures that can be learned, in order to apply such strategies over the largest possible set of domains as long as training data is available. That is, our idea is to consider algorithms that are able to binarize documents provided they have been trained for it.

In this regard, supervised learning techniques have been also studied for binarization tasks. The classification-based approach typically consists in querying every single pixel of the image, performing a feature extraction out of it and using a supervised learning algorithm to output a hypothesis about the two possible categories. Traditionally, the common choice has been to resort to Multi-Layer Perceptron~\cite{chi2001two,Hidalgo2005,kefali2014foreground}. Within this paradigm, it is obvious the consideration of deep neural networks---especially Convolutional Neural Networks (CNN) given their striking impact on the machine learning and computer vision community \cite{lecun15deep}. Some insights on the use of pixel-wise CNN for document image binarization can be found in the work of Pastor-Pellicer et al.~\cite{pastor-pellicer15insights}. The use of CNN has also proven to be robust when dealing with the binarization of document images that do not necessarily depict text information~\cite{calvo17binarization}.

Although these works have shown acceptable performance, this pixel-wise approach has two important drawbacks. The first is the high computational cost that requires labeling each pixel of the image, as it involves making as many predictions as pixels of the image. This is an important issue since binarization is usually a pre-processing step of a larger workflow, and such computational cost represents a bottleneck for the whole process. Secondly, each pixel is classified independently, without taking into account information about the labels assigned to their neighbors. That is, contextual information is somehow wasted. 

The pixel-wise approach, however, serves as an inspiration to go on considering learning-driven binarization methods. In this work we study a convolutional approach that tries to alleviate the aforementioned drawbacks. We propose the use of fully-convolutional auto-encoder models that are trained to learn a patch-wise mapping of the image to its corresponding binarization. Note that this does not consist in assigning the same label to all pixels of the input patch, but to perform a fine-grain categorization in which each pixel of the output gets a different activation value depending on whether the pixel must be labeled as background or foreground. This approach is thoroughly described in the next section.

\section{Selectional auto-encoder for document image binarization} \label{sec:method}
From a machine learning point of view, image binarization can be formulated as a two-class classification task at pixel level. The framework proposed follows this idea and, therefore, consists in learning to estimate which label must be given to every single pixel of an image. Since we are dealing with images of documents, we define the set of labels as \emph{foreground} and \emph{background}. As mentioned above, a way to implement this approach is to use a neural network to estimate which of these labels must be assigned to a pixel one by one.

In this paper we go a step further, considering an approach that uses a network topology that works in an end-to-end fashion. That is, for each input image, the model is devoted to binarizing it in just one step. This has a number of advantages such that, in this way, the classification of each pixel of the image is not computed independently, but also takes into account the label to be assigned to its neighbors. In addition, many pixels can be processed at the same time, thereby leading to higher efficiency. 

This approach is inspired by classical adaptive algorithms. Nevertheless, traditional binarization algorithms consider that the pixel value is static (its value in grayscale), whereas a different threshold is calculated depending on the local context. What the proposed framework does is to compute a different value for each pixel (neural activation) and fix a global threshold. Obviously, this activation is not obtained following a hand-crafted equation, which may serve well for a specific domain, but is learned through fully-convolutional auto-encoders.

Auto-encoders consist of feed-forward neural networks for which the input and output shape is exactly the same~\cite{hinton1994autoencoders}. The network is typically divided into two stages that learn the functions $f$ and $g$, which are called encoder and decoder functions, respectively. Traditionally, the network receives an input vector $x$ and it must minimize a divergence $L(x, g(f(x)))$. The hidden layers of the encoder perform a mapping of the input---usually decreasing its dimension---until an intermediate representation is attained. The same input is then subsequently recovered by means of the hidden layers of the decoder function. 

In their original formulation, auto-encoders are trained to learn the identity function, which might be useful as regards feature learning or dimensionality reduction because the encoder function provides a meaningful, compact representation of the input~\cite{Wang_2014}.

In this work, however, we use an auto-encoder topology for a different purpose. Here the model is trained to perform a function such that $b : \mathbb{R}^{(w \times h)} \rightarrow [0,1]^{(w \times h)}$. In other words, it learns a selectional map over a $w \times h$ image that preserves the input shape. The selectional value (activation) of each pixel depends on whether the pixel belongs to the foreground or to the background. Given the nature of the output, we call this model \emph{Selectional Auto-Encoder} ($\CAE$). This selectional encoder-decoder approach has shown promising performance in related tasks such as contour detection \cite{YangPCL016,GALLEGO2017138} but its use for document binarization remains unexplored.

Note that the approach is fully convolutional in the sense that the prediction can be done through successive convolutions and sampling operations, without any fully-connected layer. In other words, the approach just applies successive transformations to the input image. These transformations are learned through a training process, rather than being pre-established, with the aim at binarizing the document image. The training stage consists of providing examples of images and their corresponding binarized ground-truth. Since an $\CAE$ is a type of feed-forward network, the training process can be carried out by conventional means \cite{glorot2010understanding}.

Given that we are dealing with images, the hierarchy of layers of our $\CAE$ consists of a series of convolutional plus down-sampling layers, until reaching an intermediate layer in which a meaningful representation of the input is attained (function $f$). As the down-sampling layers are applied, convolutions are able to relate parts of the image that were initially far apart. It then follows a series of convolutional plus up-sampling layers that reconstruct the image up to its initial size (function $g$). The last layer consists of a set of neurons that predict a value in the range of $[0,1]$, depending on the selectional level of the corresponding input pixel. A graphical illustration of this configuration is shown in Fig.~\ref{fig:hierarchy_sae}

\begin{figure}[ht]
\centering
	\includegraphics[width=.9\columnwidth]{scheme.pdf}
	\caption{General overview of an $\CAE$ used for document image binarization (input image taken from~\cite{PratikakisZBG16}). The output layer consists of the activation level assigned to each input feature (white denotes maximum activation).}
	\label{fig:hierarchy_sae}
\end{figure} 

Thus, once the $\CAE$ has been properly trained, an image can be parsed through the network, after which a selection level is assigned to each input pixel. In practice, the network hardly outputs either $0$ or $1$ but an intermediate value. Therefore, a thresholding process is still necessary to convert the obtained neural activations into actual binary values. Those pixels whose selectional value exceeds a certain threshold are considered to belong to the foreground of the document image, whereas the others are labeled as background. The process is illustrated in Fig.~\ref{fig:binarization_cae_example}.

\begin{figure}[H]
        \centering
        \begin{subfigure}[b]{0.32\textwidth}
         \includegraphics[width=\textwidth]{example_strategy_1}
         \caption{Input image}
        \end{subfigure} 
        \begin{subfigure}[b]{0.32\textwidth}
          \includegraphics[width=\textwidth]{example_strategy_3}
          \caption{Activation}
        \end{subfigure}     
        \begin{subfigure}[b]{0.32\textwidth}
          \includegraphics[width=\textwidth]{example_strategy_4}
          \caption{Thresholding}
        \end{subfigure}
        \caption{Example of binarization task (from~\cite{PratikakisZBG16}) using an $\CAE$. A neural activation (selectional value) is obtained for each pixel of the input image, illustrated here with a heat map. A thresholding is applied in order to binarize the image.}
        \label{fig:binarization_cae_example}
\end{figure}

Quite often, the size of the input document is higher than the input layer of the $\CAE$. In those cases, it is only necessary to split the input document into pieces of the size expected by the network and parse them independently.

\subsection{Implementation details}
We have introduced previously the proposed framework to binarize documents using $\CAE$. Nevertheless, a series of implementation details need to be established to carry out this idea. 

For instance, the specific topology of an auto-encoder can be very varied. In this paper, we study three possibilities:

\begin{itemize}
\item Convolutional Auto-Encoder (CAE): traditional encoder-decoder architecture considering convolutional layers \cite{masci2011stacked}. The encoder function is implemented by three layers of convolution plus max-pooling operations, whereas the decoder function consist of three layers of convolution plus up-sampling operators.
\item Stacked What-Where Auto-Encoder (SWAEE) \cite{MaoSY16}: in this topology the pooling layers of the encoding function produce two types of connections. The \emph{what} connection is fed to the following layer, while the \emph{where} connection is fed to the analogous layer of the decoder function. We consider the instance of the SWAEE that uses a convolutional net as encoder and a deconvolutional net \cite{ZeilerKTF10} as decoder.
\item Very deep Residual Encoder-Decoder Network (RED-Net) \cite{zhao2015stacked}:  this topology includes residual connections from each encoding layer to its analogous decoding layer, which facilitates convergence and leads to better results. In addition, down-sampling is performed by convolutions with stride, instead of resorting to pooling layers. Up-sampling is achieved through deconvolutional layers.
\end{itemize}

It should be remembered that, since we are implementing a selectional approach, the network outputs are one-channel images of the same size as the input whose pixel values are restricted to the range $[0,1]$. This is achieved by considering sigmoid activations in the last layer. In addition to the topology of the network, there exist hyper-parameters (such as the size of the input layer, the number of filters per layer, the size of the filter kernels, and so on) which will be studied experimentally.

The optimization of the network weights is carried out by means of stochastic gradient descent~\cite{bottou2010large}, with a mini-batch size of $10$, and considering the adaptive learning rate proposed by Kingma and Ba \cite{kingma2014adam}. This training is performed for a maximum of $200$ epochs, with an early stop if the network does not improve during $10$ epochs. Since document binarization typically represents an imbalanced problem (more background than foreground), we make use of the loss function proposed by Pastor-Pellicer et al.~\cite{pastor2013f}, which focuses on maximizing the F-measure.  

In addition, a slight data augmentation procedure is considered so that new synthetic samples are generated by doing random scaling and flips over the original ones. Specifically, three new samples are generated from each one of the training set.
 
% =================================================================================
\section{Experiments} \label{sec:experiments} 
This section details the experimentation carried out to evaluate the discussed  approach. The performance of a binarization algorithm can be evaluated in several ways. For instance, if the algorithm is part of a workflow to perform a particular task, an interesting way to measure the performance is in relation to the final performance. However, this implies that the evaluation of the algorithm may not be totally fair, since it would be strongly related to the performance of the rest of the stages of that workflow.

In our case, we resort to the F-measure (F-m) to perform a direct evaluation of the binarization process. This metric is suitable for two-class problems in which the distribution of the samples is not balanced. Let TP, FP and FN stand for true positives (foreground pixels classified as foreground), false positives (background pixels classified as foreground), and false negatives (foreground pixels classified as background), respectively. Thus, the F-m can be computed as
$$
\text{F-m} = \frac{2 \, \text{TP}}{2 \, \text{TP} + \text{FP} + \text{FN}}.
$$

In the first series of experiments, we study the influence of both the chosen network topology and its hyper-parameters in order to select a suitable configuration for the document image binarization task. After that, we compare the goodness of the $\CAE$ approach against existing binarization methods. We also include further analysis of the performance, focusing on the errors made and the ability to adapt to different domains. 

The source code to reproduce the results that are reported below is available at \texttt{\url{https://github.com/ajgallego/document-image-binarization}}, under the conditions of the GNU General Public License version 3.

\subsection{Corpora}
In order to train the considered models and evaluate the performance of the algorithms, it is necessary to resort to ground-truth data. In this paper we are interested in validating the goodness of the binarization strategies in a wide range of document types. For this, we make use of corpora from different domains, namely those listed below:

\begin{itemize}
\item Latin text: the Document Image Binarization Contest (DIBCO) has been held for several years, starting from 2009~\cite{GatosNP09}. In each edition, the contest provides a series of images depicting Latin text that must be binarized, and the algorithms are evaluated based on a ground-truth created in a supervised way. In this work, we consider the 2014 (D14) \cite{ntirogiannis2014icfhr2014} and 2016 (D16) \cite{PratikakisZBG16} editions.\footnote{The two most recent competitions up to the date this paper was written.}
\item Palm leaf: a competition on the binarization of Balinese palm leaf manuscripts has been recently organized within the 15th International Conference on Frontiers in Handwriting Recognition \cite{burie2016icfhr2016}. The contest provided two different corpora depending on how the ground-truth was built (PL-I and PL-II).
\item Persian documents: the set of documents from the Persian Heritage Image Binarization Competition (PHI) \cite{ayatollahi2013persian} are considered here as a way of evaluating the algorithms in a different text domain.
\item Music scores: the last set of corpora consist of document images depicting music scores, which includes both music notation and text (lyrics). We specifically make use of high-resolution scans of two different old music documents: Salzinnes Antiphonal Manuscript (SAM) and Einsiedeln Stiftsbibliothek (ES), whose binarization has been studied previously~\cite{calvo17binarization}.
\end{itemize}

As example, a piece of document from each dataset is depicted in Table~\ref{tab:data_examples}.

\begin{table}[!ht]
\centering
\renewcommand{\arraystretch}{1.5}
\resizebox{1\textwidth}{!}{
\begin{tabular}{|c|c|c|}
\hline
 D14 & D16 & PL-I \& PL-II \\ \hline
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_D14.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_D16.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_PL.png} 
\end{minipage} 
\\ \hline
PHI & SAM & ES \\ \hline
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_PHI.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_SAM.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{Example_ES.png} 
\end{minipage} 
\\ \hline
\end{tabular}
}
\caption{Examples of document pieces from the considered corpora.}
\label{tab:data_examples}
\end{table}

Given that the $\CAE$ needs training data, we split the corpora in train and test partitions. In the case of DIBCO data (D14 and D16), we use in each case all the images from the rest of the contest editions as training set. PL-I and PL-II are already provided with their train and test partitions. In the case of PHI, SAM, and ES, we randomly split the corpora into train and test, with 80\% and 20\% of the available documents, respectively. A summary of the statistics of each corpus is given in Table \ref{tab:corpora}, including the number of documents and the approximate number of pixels (in millions) of each partition.

\begin{table}[ht]
\centering
\renewcommand{\arraystretch}{1.4}
\begin{tabular}{llcclcc}

\toprule[1pt]
\multirow{2}{*}{Dataset} & \multirow{2}{*}{Type} & \multicolumn{2}{c}{Documents} & & \multicolumn{2}{c}{Million pixels} \\ \cline{3-4} \cline{6-7}
 & & Train           & Test & & Train     & Test \\ \hline
D14         & Latin text            & 76 & 10 & & 85 & 10  \\
D16         & Latin text            & 76 & 10 & & 82 & 13 \\ 
PL-I        & Palm leaf             & 50 & 50 & & 125 & 125 \\ 
PL-II       & Palm leaf             & 50 & 50 & & 125 & 125 \\ 
PHI         & Persian               & 12 & 3 & & 16 & 2 \\ 
ES          & Music score           & 8 & 2 & & 162 & 41 \\ 
SAM         & Music score           & 8 & 2 & & 131 & 33 \\
\bottomrule[1pt]
\end{tabular}
\caption{Corpora considered in our experiments.}
\label{tab:corpora}
\end{table}


\subsection{Model selection}
As discussed earlier, this work focuses on the binarization approach of documents using $\CAE$. The main emphasis is on the approach itself and not so much on finding the best configuration of the model. However, it is obvious that we need to establish a specific configuration that allows us to evaluate the capabilities of the approach. This is why we want to study how the different topologies and hyper-parameters of the models affect the performance. 

To carry out this experiment, we are going to restrict ourselves to use only D16 dataset. The reason is twofold: on the one hand, Latin text binarization from DIBCO competitions is a standard benchmark for evaluating binarization strategies, so D16 is a good representative of the task at hand; on the other hand, we do not want our final results depending on an excessive tuning of the model that best fits. Our premise is that the goodness of the $\CAE$ approach lies in its formulation, and not in selecting the optimum configuration for every case. We therefore assume that the best configuration in this study will be a good representative of the $\CAE$ approach for document image binarization. 

Since the general configuration of the models has already been specified previously, we only study three hyper-parameters: the size of the input/output window (patch of the document that is processed in a single step), the number of convolutional filters in each layer, and the kernel size of the convolutional filters.

The first aspect to consider is the size of the window, fixing $64$ filters per convolution and kernel size of $3 \times 3$. In order to reduce the search space, we have restricted ourselves to square windows of sides $64$, $128$, $256$, and $384$. Figure~\ref{fig:wsize} shows that this parameter can strongly influence the performance of the model. For instance, it is observed that RED-Net achieves the highest F-m when considering windows of side $256$, degrading significantly its performance with windows of side $384$. Also, the SWWAE obtains the worst performance with $64 \times 64$ windows, but its results increase noticeably with bigger windows. The CAE model reports a more robust behavior with respect to this parameter, although its results are generally below the other configurations.

\begin{figure}[ht]
\centering
	\includegraphics[width=.7\columnwidth]{wsize.pdf}
	\caption{Influence in terms of F-m of the side of the square window with respect to each considered $\CAE$ model.}
	\label{fig:wsize}
\end{figure} 

The second hyper-parameter to study is the number of filters per convolutional layer. In spite of knowing that it is not necessarily optimal, let us force that all convolutional layers consist of the same number of filters so that this study does not become huge. Specifically, we consider $16$, $32$, $64$, $96$, and $128$ filters per layer. Figure~\ref{fig:fsize} shows the curves of this experiment. In this case, the number of filters has a smaller influence on RED-Net but greater on SWWAE and CAE. In the SWWAE model, it is relevant because it varies from less than $87$ (16 filters) to $89.3$ (64 filters) of F-m. Again, the best value is attained for RED-Net, specifically with $64$ filters.

\begin{figure}[ht]
\centering
	\includegraphics[width=.7\columnwidth]{fsize.pdf}
	\caption{Influence in terms of F-m of the number of filters per convolution with respect to each considered $\CAE$ model.}
	\label{fig:fsize}
\end{figure} 

The last model hyper-parameter to study is the kernel size of the convolutional operators. As in the previous case, we consider the same number for all layers:  square kernels of side $3$, $5$, or $7$. As illustrated in Fig.~\ref{fig:ksize}, both RED-Net and SWWAE are hardly sensitive to this parameter. It can be seen that their curves are almost straight, albeit with a negligible improvement as the value increases in the case of RED-Net. The kernel size seems to be somewhat relevant for CAE, as its performance slightly improves with values higher than $3$. 

\begin{figure}[ht]
\centering
	\includegraphics[width=.7\columnwidth]{ksize.pdf}
	\caption{Influence in terms of F-m of the size of the convolutional kernel with respect to each considered $\CAE$ model.}
	\label{fig:ksize}
\end{figure} 

Generally, it can be seen that the $\CAE$ approach itself is quite robust beyond configurations and hyper-parameters, as the results only range from $88$ to $91$ of F-m. In particular, the RED-Net model with $256 \times 256$, $64$ filters, and $7 \times 7$ kernels attains the best performance amongst the evaluated possibilities. However, we finally selected a kernel size of $5 \times 5$ given that $7 \times 7$ kernels implies a much larger network with an insignificant improvement. From here on, we shall consider this full configuration as a representative of the $\CAE$ approach for document binarization.

After selecting this model, we also studied the influence of the threshold used to convert the selectional values (neural activations) into the expected binary output. We measured the F-m obtained for different threshold values---within the range of $[0.1,0.9]$. It was observed that the variability of the performance with respect to this parameter is negligible (at most, less than 0.1). This fact, therefore, reinforces the robustness of the activations predicted by the $\CAE$ because they tend to be close to either $0$ or $1$, and thereby decreasing the importance of the threshold. 

\subsection{Comparative assessment}
In this section we compare the performance of the $\CAE$ approach against other binarization algorithms, including a baseline reference as well as classical and state-of-the-art algorithms. Specifically, we consider the following methods:

\begin{itemize}
\item Baseline: Otsu~\cite{otsu1975threshold} and Niblack~\cite{niblack1985introduction}. Although these methods are old, they still remain as good baseline references.
\item Classic: Sauvola~\cite{sauvola2000adaptive}, Wolf et al.~\cite{wolf02text}, and Gatos~\cite{gatos06adaptive}. These strategies have been traditionally used to binarize document images in many existing workflows.
\item State of the art: Su et al.~\cite{su2013robust}, Sauvola MS (Lazzara and Geraud \cite{lazzara2014efficient}), Howe~\cite{howe2013document}, and Kliger and Tal~\cite{PratikakisZBG16}. Recent methods that constitute the state of the art for document image binarization.
\end{itemize}

In addition, we include the pixel-wise CNN approach studied by Pastor-Pellicer et al.~\cite{pastor-pellicer15insights} as a representative of learning-based binarization with neural networks. All these strategies have been introduced previously in more detail (cf. Section~\ref{sec:background}). 

Furthermore, there exist two possibilities to train the selected $\CAE$ model that we analyze empirically: (i) training the model only with the specific train partition of each dataset ($\CAE$ specific); and (ii) training the model with the train partitions of all datasets ($\CAE$ global). Thus, we evaluate if the $\CAE$ benefits more from the suitability of the data or from its quantity.

Table~\ref{tab:results_comparison} presents the figures obtained in terms of F-m for each corpus. An initial remark to begin with is that there is a noticeable difference amongst the performance attained by the different strategies in each dataset, reinforcing our idea of including as many types of document as possible in these experiments.

\begin{table}
\centering
\renewcommand{\arraystretch}{1.4}
\resizebox{1\textwidth}{!}{
\begin{tabular}{lcccccccc}
\toprule[1pt]
			 & \multicolumn{7}{c}{Dataset} \\ \cline{2-8}
Method 	 	  	& D14 & D16 & PL-I & PL-II & PHI & ES & SAM & Avg. \\ \hline
Otsu 			& 91.56 & 73.79 & 34.92 & 34.75 & 90.77 & 74.32 & 79.72 & 68.55 \\
Niblack 		& 22.26 & 16.7  & 16.25 & 16.15 & 25.70 & 34.3  & 31.96 & 23.33 \\
Sauvola 		& 77.08 & 82.00	& 9.94  & 9.75  & 81.64 & 61.69 & 65.14 & 55.32 \\
Wolf et al.		& 90.47 & 81.76 & 30.72 & 30.30 & \underline{91.84} & 67.02 & 77.00 & 67.02 \\
Gatos 			& 91.97 & 74.97 & 19.75 & 19.88 & 91.69 & 74.63 & 80.11 & 64.71 \\
Sauvola MS 		& 87.86 & 65.04 & 42.76 & 42.41 & 91.39 & 74.00 & 79.79 & 69.03 \\
Su et al.		& \underline{95.14} & 90.27 & 48.44 & 48.04 & 89.27 & 75.61 & 81.55 & \underline{75.48} \\
Howe 			& 90.00 & 80.64 & 46.49 & 45.77 & 88.97 & 76.03 & 80.52 & 72.63 \\
Kliger and Tal 	& 95.00 & \underline{90.48} & 46.58 & 45.88 & 89.67 & \underline{76.88} & \underline{82.09} & 75.22 \\
CNN 			& 81.23 & 54.58 & \underline{51.47} & \underline{51.09} & 88.53 & 75.81 & 81.05 & 69.11  \\ \hline
$\CAE$ specific		& \textbf{98.05} & \textbf{91.65} & \textbf{69.65} & \textbf{69.13} & \textbf{93.82} & \textbf{78.45} & \textbf{83.13} & \textbf{83.41} \\
$\CAE$ global	& 89.12 & 85.27 & 63.74 & 63.21 & 71.63 & 77.11 & 81.38 & 75.92 \\
  \bottomrule[1pt]
\end{tabular}
}
\caption{Performance comparison of the $\CAE$ approach against existing binarization strategies in terms of F-m. Values in bold represent the best performance in each dataset; values underlined represent the best performance in each dataset without considering the $\CAE$ approach.}
\label{tab:results_comparison}
\end{table}

In general, baseline and classical methods get worse performance, with some exceptional cases that depend on the method and the dataset. This can be seen in the irregularity of methods such as Gatos or Otsu, which achieve good results in D14, PHI, ES or SAM but behave poorly in P-I and P-II. Furthermore, it is observed that state-of-the-art methods obtain a higher average performance (\emph{Avg.} column) than the previous ones. In particular, the method proposed by Kliger and Tal obtains the best result among the existing methods in three of the seven datasets considered, although the method proposed by Su et al. attains the best results on average. The learning-based approach with CNN depicts a quite irregular behavior, which ranges from being the best method within the existing ones in two datasets (P-I and P-II) to not outperforming the specialized level of other methods with text documents. However, it is closer to the classical ones than to the considered state-of-the-art methods, on average.

Regarding the $\CAE$ approach, it is observed that it is better to train the model with specialized data since results are always superior to the case of training with all available data. In the former case, the performance is able to improve all the figures obtained by existing algorithms, whereas in the latter many of them are improved as well. It can be seen that the improvement is especially remarkable in those datasets where the existing methods find more difficulties (\emph{ie.}, P-I and P-II). Focusing on the average case, the studied approach is able to increase the state of the art from $75.48$ to $83.41$ ($\CAE$ specific) or to $75.92$ ($\CAE$ global) of F-m. 

For all the above, the goodness of the $\CAE$ approach is demonstrated given that it represents an effective document image binarization method based on machine learning that is able to outperform the state of the art.

\subsection{Performance analysis}
In this section the performance of the approach is analyzed, with special attention to the errors produced.

First, it is interesting to show a qualitative reference of the errors produced by the $\CAE$-based binarization. For this, we have selected datasets of different types with the highest error, namely D16, PL-I and ES. Table~\ref{tab:error_output} shows a series of representative examples of the performance obtained in these corpora, as well as where and how errors have occurred. For the sake of comparison, we include the state of the art in each dataset (Kliger and Tal in D16 and ES; CNN in PL-I), as well as that of Su et al. because of reporting the best average.

\begin{table}[!ht]
\centering
\renewcommand{\arraystretch}{1.4}
\resizebox{1\textwidth}{!}{
\begin{tabular}{|c|c|c|c|}
\hline
 & D16 & PL-I & ES \\ \hline
\begin{minipage}[c]{0.1\textwidth}
\centering
Source
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{d16_error_src.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{palm_error_src.png} 
\end{minipage} & 
\begin{minipage}[c]{0.34\textwidth} 
\includegraphics[width=1\columnwidth]{es_error_src.png} 
\end{minipage} 
\\ \hline
\begin{minipage}[c]{0.1\textwidth}
\centering
$\CAE$
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{d16_error_sae.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{palm_error_sae.png} 
\end{minipage} & 
\begin{minipage}[c]{0.34\textwidth} 
\includegraphics[width=1\columnwidth]{es_error_sae.png} 
\end{minipage} 
\\ \hline
\begin{minipage}[c]{0.1\textwidth}
\centering
State of the art
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{d16_error_kliger.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{palm_error_cnn.png} 
\end{minipage} & 
\begin{minipage}[c]{0.34\textwidth} 
\includegraphics[width=1\columnwidth]{es_error_kliger.png} 
\end{minipage} 
\\ \hline
\begin{minipage}[c]{0.1\textwidth}
\centering
Su et al.
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{d16_error_su.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{palm_error_su.png} 
\end{minipage} & 
\begin{minipage}[c]{0.34\textwidth} 
\includegraphics[width=1\columnwidth]{es_error_su.png} 
\end{minipage} 
\\ \hline

\end{tabular}
}
\caption{Qualitative reference of the errors produced when binarizing documents of D16, PL-I, and ES with $\CAE$, the state of the art, and Su et al. Coloring: foreground pixels identified as foreground in black, background pixels identified as background in white, foreground pixels identified as background in red, and background pixels identified as foreground in blue.}
\label{tab:error_output}
\end{table}

This table illustrates the location of the improvement obtained by the $\CAE$ in comparison to existing methods. Generally, it is much more effective when discarding false negatives (blue pixels), while maintaining a similar level in the detection of true positives (black pixels). That is, the approach is especially beneficial when dealing with the background of complex documents. 

Furthermore, given the underlying operation of the $\CAE$, it seems obvious to think that the errors may occur more often in the regions of the image that are closer to the edges of the considered input windows, since it is where there is less context of the neighborhood. To study this possibility, we computed a heat map of the the position of the errors with regard to the output window. We observed that errors are distributed without any defined pattern in most of the datasets, with some exceptions for palm leaf and music score documents. That is why we study the general case with D14, and the particular cases with PL-I and SAM as representatives of palm leaf and music scores, respectively.

Graphical results of this study are shown in Table \ref{tab:heatmaps}. In addition to the aforementioned heat maps (error heat map), we have included another heat map that illustrates the foreground positions in the ground-truth (ground-truth heat map) and a representative ground-truth output window.

\begin{table}[!ht]
\centering
\renewcommand{\arraystretch}{1.7}
\resizebox{1\textwidth}{!}{
\begin{tabular}{|c|c|c|c|}
\hline
 & D14 & PL-I & SAM \\ \hline
\begin{minipage}[c]{0.2\textwidth}
\centering
$\CAE$ error heat map
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{error_heat_d14.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{error_heat_palm.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{error_heat_sal.png} 
\end{minipage} 
\\ \hline
\begin{minipage}[c]{0.2\textwidth}
\centering
Ground-truth foreground heat map
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{gt_heat_d14.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{gt_heat_palm.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{gt_heat_sal.png} 
\end{minipage} 
\\ \hline
\begin{minipage}[c]{0.2\textwidth}
\centering
Example of ground-truth window
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{heat_ex_d14.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{heat_ex_pl.png} 
\end{minipage} & 
\begin{minipage}[c]{0.3\textwidth} 
\includegraphics[width=1\columnwidth]{heat_ex_sam.png} 
\end{minipage} 
\\ \hline
\end{tabular}
}
\caption{Heat map (\%) of the errors ($\CAE$ error heat map) and the foreground ground-truth pixels (ground-truth foreground heat map) with respect to positions within the  input/output window. An example of ground-truth output window is also included.}
\label{tab:heatmaps}
\end{table}

As can be observed in all cases, the areas of the image where the errors are concentrated do not have a direct relationship with their position within the window, but with the regions where the ink pixels are concentrated in the corresponding dataset. In the case of D14, as ink is distributed evenly across all positions, the error map does not depict any regular pattern. In the case of PL-I, the window is large enough to hold two text lines, so the ink map shows these two lines; accordingly, the error also seems to draw these two lines because errors are mainly produced when detecting ink (as studied previously). The case of SAM is similar: the ink map shows the four staff lines that fit in an input window, with a concentration of music symbols in the top ones. This is somewhat depicted in the error heat map as well; however, the error obtained by the $\CAE$ in this dataset is lower and, therefore, the error heat map is not so intense.

\subsection{Domain adaptation}
We have seen previously that an $\CAE$ is able to improve state-of-the-art binarization when the training set contains samples of the application domain. We study in this section the performance of the $\CAE$ when it is applied to document types of which no examples are provided during training.

This experiment is performed by checking how a model specifically trained for one  dataset behaves in the rest of the corpora. Table \ref{tab:domain_adaptation} shows this study, in which rows represent the training partitions and columns represent the test partitions. In this sense, we ignore the comparisons between D14 and D16 because it does not make sense to the objective of this experiment given the composition of their train/test partitions.

\begin{table}[ht]
\centering
\renewcommand{\arraystretch}{1.4}
\begin{tabular}{lccccccccc}
\toprule[1pt]
\multirow{2}{*}{Training}	&		 & \multicolumn{7}{c}{Test} & \\ \cline{3-9}
        	 & & D14 & D16 & PL-I & PL-II & PHI & ES & SAM & Avg.\\ \hline
D14          & &  98.05	& --	& 51.09 & 50.51	& 89.45 & 76.28	& 81.50 & 74.48 \\
D16          & &   -- &	91.65 &	47.22 & 	46.59 &	87.04 &	76.10 &	81.73 & 71.72 \\
PL-I & & 32.90 &	15.40	&69.65&	69.33&	52.51	&0.23&	0.08 &	34.30 \\
PL-II & & 20.94	 & 10.52 & 69.46 & 69.13 & 50.23 & 0.54 & 0.24 & 31.58\\
PHI	  & & 88.75 & 72.17 & 42.12 & 41.89 & 93.82 & 73.20 & 77.42 & 69.91 \\
ES           & & 85.46 &	63.55	& 41.72 &	41.49 &	81.20	& 78.45 &	80.60 & 67.49 \\
SAM          & & 80.03 & 41.16 &	12.81 &	12.74 &	18.99 &	76.10 & 83.13 &	46.42 \\
\bottomrule[1pt]
\end{tabular}
\caption{Performance in terms of F-m achieved by an $\CAE$ trained with a specific training partition (rows) on the test partitions of the rest of datasets (columns).}
\label{tab:domain_adaptation}
\end{table}

These results give insights on the sensitivity of the model with respect to the similarity of the train and test domains. It can be observed that results are good when the $\CAE$ is trained with data of similar domains. This is the case of results amongst text (D14, D16, and PHI), palm leaf (PL-I and PL-II), and musical scores (SAM and ES) documents. Likewise, this sensitivity is confirmed with the results obtained when models are trained with palm leaf documents (the most different to the rest), with which very poor results are achieved in the other domains.

It is interesting to remark that the most general documents are the ones depicting Latin text (D14 and D16). Specifically, the model trained for D14 obtains a fair average performance considering all datasets, which is competitive against existing binarization algorithms (cf. Table \ref{tab:results_comparison}).

\section{Conclusions} \label{sec:conclusions}
In this paper an approach for document image binarization has been presented. The strategy is to train a Selectional Auto-Encoder ($\CAE$) that is able to learn an end-to-end transformation to binarize an image. Given a piece of image of a fixed size, the model outputs a selectional value for each pixel of the image depending on the confidence whether the pixel belongs to the foreground of the document. These values are eventually thresholded to yield a discrete binary result.

The influence of the model configuration and hyper-parameters has been studied preliminary. Then, the selected model has been evaluated under several datasets of distinct typology, as well as different ways of using the training set: using only domain-specific data or using all available data. The former option reported better results, yet the latter one also achieved a fair performance. 

The studied approach has been faced against both classical and state-of-the-art binarization algorithms. Results reported that the $\CAE$ approach is able to outperform the rest of binarization algorithms, increasing the best average performance obtained by existing strategies from $75.48$ up to $83.41$ in terms of F-measure.

The errors produced by the selected $\CAE$ have been analyzed in detail, drawing the conclusion that the approach detects the background very accurately and errors are concentrated around foreground strokes. Furthermore, it has been shown that errors are not directly related to their position within the input/output window.

Finally, a domain adaptation experiment has been carried out, in which we measured the performance of the models trained specifically for a dataset in the rest of them. The main conclusion drawn from this experiment is that the performance of the $\CAE$ is directly related to the similarity between train and test data. That is, good results are obtained if an appropriate training set is available for that domain.

As prospects of future work, we want to study how to include a priori information in the binarization model. For instance, it would be interesting to favor the continuity of ink strokes, so that it is more unlikely to find broken shapes in the foreground. In addition, in those cases when documents are too large to fit into a single window, it may be helpful to connect all the independent processes through recurrent connections so that information from all areas of the document are taken into account at once.

\section*{Acknowledgments}
This work was partially supported by the Social Sciences and Humanities Research Council of Canada, and the Universidad de Alicante through grant GRE-16-04.


\bibliographystyle{elsarticle-num}
\bibliography{binarization}

\end{document}

binarization.bbl
\begin{thebibliography}{10}
\expandafter\ifx\csname url\endcsname\relax
  \def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
\expandafter\ifx\csname href\endcsname\relax
  \def\href#1#2{#2} \def\path#1{#1}\fi

\bibitem{LOULOUDIS20083758}
G.~Louloudis, B.~Gatos, I.~Pratikakis, C.~Halatsis, Text line detection in
  handwritten documents, Pattern Recognition 41~(12) (2008) 3758--3772.

\bibitem{saabni2014text}
R.~Saabni, A.~Asi, J.~El-Sana, Text line extraction for historical document
  images, Pattern Recognition Letters 35 (2014) 23--33.

\bibitem{HE20154036}
S.~He, M.~Wiering, L.~Schomaker, Junction detection in handwritten documents
  and its application to writer identification, Pattern Recognition 48~(12)
  (2015) 4036--4048.

\bibitem{calvo2015avoiding}
J.~Calvo-Zaragoza, I.~Barbancho, L.~J. Tard{\'o}n, A.~M. Barbancho, Avoiding
  staff removal stage in optical music recognition: application to scores
  written in white mensural notation, Pattern Analysis and Applications 18~(4)
  (2015) 933--943.

\bibitem{GIOTIS2017310}
A.~P. Giotis, G.~Sfikas, B.~Gatos, C.~Nikou, A survey of document image word
  spotting techniques, Pattern Recognition 68 (2017) 310 -- 332.

\bibitem{zeiler2014visualizing}
M.~D. Zeiler, R.~Fergus, Visualizing and understanding convolutional networks,
  in: European conference on computer vision, Springer, 2014, pp. 818--833.

\bibitem{lecun15deep}
Y.~LeCun, Y.~Bengio, G.~Hinton, {Deep learning}, Nature 521~(7553) (2015)
  436--444.

\bibitem{otsu1975threshold}
N.~Otsu, A threshold selection method from gray-level histograms, Automatica
  11~(285-296) (1975) 23--27.

\bibitem{niblack1985introduction}
W.~Niblack, An introduction to digital image processing, Strandberg Publishing
  Company, 1985.

\bibitem{sauvola2000adaptive}
J.~Sauvola, M.~Pietik{\"a}inen, Adaptive document image binarization, Pattern
  Recognition 33~(2) (2000) 225--236.

\bibitem{wolf02text}
C.~Wolf, J.~M. Jolion, F.~Chassaing, Text localization, enhancement and
  binarization in multimedia documents, in: Proceedings of the {I}nternational
  {C}onference on {P}attern {R}ecognition, Vol.~2, 2002, pp. 1037--1040.

\bibitem{lazzara2014efficient}
G.~Lazzara, T.~G{\'e}raud, Efficient multiscale sauvola?s binarization,
  International Journal on Document Analysis and Recognition (IJDAR) 17~(2)
  (2014) 105--123.

\bibitem{gatos06adaptive}
B.~Gatos, I.~Pratikakis, S.~J. Perantonis, Adaptive degraded document image
  binarization, {Pattern Recognition} 39~(3) (2006) 317--327.

\bibitem{su2013robust}
B.~Su, S.~Lu, C.~L. Tan, Robust document image binarization technique for
  degraded document images, IEEE transactions on image processing 22~(4) (2013)
  1408--1417.

\bibitem{howe2011laplacian}
N.~R. Howe, A laplacian energy for document binarization, in: 2011
  International Conference on Document Analysis and Recognition (ICDAR), IEEE,
  2011, pp. 6--10.

\bibitem{howe2013document}
N.~R. Howe, Document binarization with automatic parameter tuning,
  International Journal on Document Analysis and Recognition (IJDAR) 16~(3)
  (2013) 247--258.

\bibitem{katz2007direct}
S.~Katz, A.~Tal, R.~Basri, Direct visibility of point sets, in: ACM
  Transactions on Graphics (TOG), Vol.~26, ACM, 2007, p.~24.

\bibitem{PratikakisZBG16}
I.~Pratikakis, K.~Zagoris, G.~Barlas, B.~Gatos, {ICFHR2016} handwritten
  document image binarization contest {(H-DIBCO} 2016), in: 15th International
  Conference on Frontiers in Handwriting Recognition, {ICFHR} 2016, Shenzhen,
  China, October 23-26, 2016, 2016, pp. 619--623.

\bibitem{chi2001two}
Z.~Chi, K.~W. Wong, A two-stage binarization approach for document images, in:
  Proceedings of the International Symposium on Intelligent Multimedia, Video
  and Speech Processing, IEEE, 2001, pp. 275--278.

\bibitem{Hidalgo2005}
J.~L. Hidalgo, S.~Espa{\~{n}}a, M.~J. Castro, J.~A. P{\'e}rez, Enhancement and
  cleaning of handwritten data by using neural networks, in: Proceedings of the
  2nd Iberian Conference on Pattern Recognition and Image Analysis, Springer
  Berlin Heidelberg, Berlin, Heidelberg, 2005, pp. 376--383.

\bibitem{kefali2014foreground}
A.~Kefali, T.~Sari, H.~Bahi, {Foreground-back\-ground separation by feed
  forward neural networks in old manuscripts}, Informatica 38~(4).

\bibitem{pastor-pellicer15insights}
J.~Pastor{-}Pellicer, S.~E. Boquera, F.~Zamora{-}Mart{\'{\i}}nez, M.~Z. Afzal,
  M.~J.~C. Bleda, Insights on the use of convolutional neural networks for
  document image binarization, in: Advances in Computational Intelligence -
  13th International Work-Conference on Artificial Neural Networks, {IWANN}
  2015, Palma de Mallorca, Spain, June 10-12, 2015. Proceedings, Part {II},
  2015, pp. 115--126.

\bibitem{calvo17binarization}
J.~Calvo-Zaragoza, G.~Vigliensoni, I.~Fujinaga, Pixel-wise binarization of
  musical documents with convolutional neural networks, in: Fifteenth IAPR
  International Conference on Machine Vision Applications, 2017, pp. 362--5.

\bibitem{hinton1994autoencoders}
G.~E. Hinton, R.~S. Zemel, Autoencoders, minimum description length and
  helmholtz free energy, in: Advances in Neural Information Processing Systems,
  1994, pp. 3--10.

\bibitem{Wang_2014}
W.~Wang, Y.~Huang, Y.~Wang, L.~Wang, Generalized autoencoder: A neural network
  framework for dimensionality reduction, in: The IEEE Conference on Computer
  Vision and Pattern Recognition (CVPR) Workshops, 2014, pp. 490--497.

\bibitem{YangPCL016}
J.~Yang, B.~L. Price, S.~Cohen, H.~Lee, M.~Yang, Object contour detection with
  a fully convolutional encoder-decoder network, in: 2016 {IEEE} Conference on
  Computer Vision and Pattern Recognition, {CVPR} 2016, Las Vegas, NV, USA,
  June 27-30, 2016, 2016, pp. 193--202.

\bibitem{GALLEGO2017138}
A.-J. Gallego, J.~Calvo-Zaragoza, Staff-line removal with selectional
  auto-encoders, {Expert Systems with Applications} 89 (2017) 138--148.

\bibitem{glorot2010understanding}
X.~Glorot, Y.~Bengio, Understanding the difficulty of training deep feedforward
  neural networks, in: Proceedings of the Thirteenth International Conference
  on Artificial Intelligence and Statistics, 2010, pp. 249--256.

\bibitem{masci2011stacked}
J.~Masci, U.~Meier, D.~Cire{\c{s}}an, J.~Schmidhuber, Stacked convolutional
  auto-encoders for hierarchical feature extraction, Artificial Neural Networks
  and Machine Learning--ICANN 2011 (2011) 52--59.

\bibitem{MaoSY16}
X.~Mao, C.~Shen, Y.~Yang, Image restoration using very deep convolutional
  encoder-decoder networks with symmetric skip connections, in: Advances in
  Neural Information Processing Systems 29: Annual Conference on Neural
  Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain,
  2016, pp. 2802--2810.

\bibitem{ZeilerKTF10}
M.~D. Zeiler, D.~Krishnan, G.~W. Taylor, R.~Fergus, Deconvolutional networks,
  in: The Twenty-Third {IEEE} Conference on Computer Vision and Pattern
  Recognition, {CVPR} 2010, San Francisco, CA, USA, 13-18 June 2010, 2010, pp.
  2528--2535.

\bibitem{zhao2015stacked}
J.~Zhao, M.~Mathieu, R.~Goroshin, Y.~LeCun, Stacked what-where auto-encoders,
  arXiv preprint arXiv:1506.02351.

\bibitem{bottou2010large}
L.~Bottou, Large-scale machine learning with stochastic gradient descent, in:
  Proceedings of COMPSTAT'2010, Springer, 2010, pp. 177--186.

\bibitem{kingma2014adam}
D.~Kingma, J.~Ba, Adam: A method for stochastic optimization, arXiv preprint
  arXiv:1412.6980.

\bibitem{pastor2013f}
J.~Pastor-Pellicer, F.~Zamora-Mart{\'\i}nez, S.~Espa{\~n}a-Boquera, M.~J.
  Castro-Bleda, F-measure as the error function to train neural networks, in:
  International Work-Conference on Artificial Neural Networks, Springer, 2013,
  pp. 376--384.

\bibitem{GatosNP09}
B.~Gatos, K.~Ntirogiannis, I.~Pratikakis, {ICDAR} 2009 document image
  binarization contest {(DIBCO} 2009), in: 10th International Conference on
  Document Analysis and Recognition, {ICDAR} 2009, Barcelona, Spain, 26-29 July
  2009, 2009, pp. 1375--1382.

\bibitem{ntirogiannis2014icfhr2014}
K.~Ntirogiannis, B.~Gatos, I.~Pratikakis, Icfhr2014 competition on handwritten
  document image binarization (h-dibco 2014), in: Frontiers in Handwriting
  Recognition (ICFHR), 2014 14th International Conference on, IEEE, 2014, pp.
  809--813.

\bibitem{burie2016icfhr2016}
J.-C. Burie, M.~Coustaty, S.~Hadi, M.~W.~A. Kesiman, J.-M. Ogier, E.~Paulus,
  K.~Sok, I.~M.~G. Sunarya, D.~Valy, Icfhr2016 competition on the analysis of
  handwritten text in images of balinese palm leaf manuscripts, in: Frontiers
  in Handwriting Recognition (ICFHR), 2016 15th International Conference on,
  IEEE, 2016, pp. 596--601.

\bibitem{ayatollahi2013persian}
S.~M. Ayatollahi, H.~Z. Nafchi, {Persian heritage image binarization
  competition (PHIBC 2012)}, in: Pattern Recognition and Image Analysis (PRIA),
  2013 First Iranian Conference on, IEEE, 2013, pp. 1--4.

\end{thebibliography}
