  In this paper, we present a novel error measure to compare a segmentation
against ground truth. This measure, which we call Tolerant Edit Distance (TED),
is motivated by two observations: (1) Some errors, like small boundary shifts,
are tolerable in practice. Which errors are tolerable is application dependent
and should be a parameter of the measure. (2) Non-tolerable errors have to be
corrected manually. The time needed to do so should be reflected by the error
measure. Using integer linear programming, the TED finds the minimal weighted
sum of split and merge errors exceeding a given tolerance criterion, and thus
provides a time-to-fix estimate. In contrast to commonly used measures like
Rand index or variation of information, the TED (1) does not count small, but
tolerable, differences, (2) provides intuitive numbers, (3) gives a time-to-fix
estimate, and (4) can localize and classify the type of errors. By supporting
both isotropic and anisotropic volumes and having a flexible tolerance
criterion, the TED can be adapted to different requirements. On example
segmentations for 3D neuron segmentation, we demonstrate that the TED is
capable of counting topological errors, while ignoring small boundary shifts.
