  This paper presents an open tool for standardizing the evaluation process of
the layout analysis task of document images at pixel level. We introduce a new
evaluation tool that is both available as a standalone Java application and as
a RESTful web service. This evaluation tool is free and open-source in order to
be a common tool that anyone can use and contribute to. It aims at providing as
many metrics as possible to investigate layout analysis predictions, and also
provide an easy way of visualizing the results. This tool evaluates document
segmentation at pixel level, and support multi-labeled pixel ground truth.
Finally, this tool has been successfully used for the ICDAR2017 competition on
Layout Analysis for Challenging Medieval Manuscripts.
