  Despite the recent success of text detection and recognition methods,
existing evaluation metrics fail to provide a fair and reliable comparison
among those methods. In addition, there exists no end-to-end evaluation metric
that takes characteristics of OCR tasks into account. Previous end-to-end
metric contains cascaded errors from the binary scoring process applied in both
detection and recognition tasks. Ignoring partially correct results raises a
gap between quantitative and qualitative analysis, and prevents fine-grained
assessment. Based on the fact that character is a key element of text, we
hereby propose a Character-Level Evaluation metric (CLEval). In CLEval, the
\textit{instance matching} process handles split and merge detection cases, and
the \textit{scoring process} conducts character-level evaluation. By
aggregating character-level scores, the CLEval metric provides a fine-grained
evaluation of end-to-end results composed of the detection and recognition as
well as individual evaluations for each module from the end-performance
perspective. We believe that our metrics can play a key role in developing and
analyzing state-of-the-art text detection and recognition methods. The
evaluation code is publicly available at https://github.com/clovaai/CLEval.
