  Recently, significant progress has been made applying machine learning to the
problem of table structure inference and extraction from unstructured
documents. However, one of the greatest challenges remains the creation of
datasets with complete, unambiguous ground truth at scale. To address this, we
develop a new, more comprehensive dataset for table extraction, called
PubTables-1M. PubTables-1M contains nearly one million tables from scientific
articles, supports multiple input modalities, and contains detailed header and
location information for table structures, making it useful for a wide variety
of modeling approaches. It also addresses a significant source of ground truth
inconsistency observed in prior datasets called oversegmentation, using a novel
canonicalization procedure. We demonstrate that these improvements lead to a
significant increase in training performance and a more reliable estimate of
model performance at evaluation for table structure recognition. Further, we
show that transformer-based object detection models trained on PubTables-1M
produce excellent results for all three tasks of detection, structure
recognition, and functional analysis without the need for any special
customization for these tasks. Data and code will be released at
https://github.com/microsoft/table-transformer.
