  In this paper, we propose a new class of metric for table structure
recognition (TSR) evaluation, called grid table similarity (GriTS). Unlike
prior metrics, GriTS evaluates the correctness of a predicted table directly in
its natural form as a matrix. To create a similarity measure between matrices,
we generalize the two-dimensional largest common substructure (2D-LCS) problem,
which is NP-hard, to the 2D most similar substructures (2D-MSS) problem and
propose a polynomial-time heuristic for solving it. This algorithm produces
both an upper and a lower bound on the true similarity between matrices. We
show using evaluation on a large real-world dataset that in practice there is
almost no difference between these bounds. We compare GriTS to other metrics
and empirically validate that matrix similarity exhibits more desirable
behavior than alternatives for TSR performance evaluation. Finally, GriTS
unifies all three subtasks of cell topology recognition, cell location
recognition, and cell content recognition within the same framework, which
simplifies the evaluation and enables more meaningful comparisons across
different types of TSR approaches. Code will be released at
https://github.com/microsoft/table-transformer.
