  We present a new table structure recognition (TSR) approach, called
TSRFormer, to robustly recognizing the structures of complex tables with
geometrical distortions from various table images. Unlike previous methods, we
formulate table separation line prediction as a line regression problem instead
of an image segmentation problem and propose a new two-stage DETR based
separator prediction approach, dubbed \textbf{Sep}arator \textbf{RE}gression
\textbf{TR}ansformer (SepRETR), to predict separation lines from table images
directly. To make the two-stage DETR framework work efficiently and effectively
for the separation line prediction task, we propose two improvements: 1) A
prior-enhanced matching strategy to solve the slow convergence issue of DETR;
2) A new cross attention module to sample features from a high-resolution
convolutional feature map directly so that high localization accuracy is
achieved with low computational cost. After separation line prediction, a
simple relation network based cell merging module is used to recover spanning
cells. With these new techniques, our TSRFormer achieves state-of-the-art
performance on several benchmark datasets, including SciTSR, PubTabNet and WTW.
Furthermore, we have validated the robustness of our approach to tables with
complex structures, borderless cells, large blank spaces, empty or spanning
cells as well as distorted or even curved shapes on a more challenging
real-world in-house dataset.
