  Tables contain factual and quantitative data accompanied by various
structures and contents that pose challenges for machine comprehension.
Previous methods generally design task-specific architectures and objectives
for individual tasks, resulting in modal isolation and intricate workflows. In
this paper, we present a novel large vision-language model, TabPedia, equipped
with a concept synergy mechanism. In this mechanism, all the involved diverse
visual table understanding (VTU) tasks and multi-source visual embeddings are
abstracted as concepts. This unified framework allows TabPedia to seamlessly
integrate VTU tasks, such as table detection, table structure recognition,
table querying, and table question answering, by leveraging the capabilities of
large language models (LLMs). Moreover, the concept synergy mechanism enables
table perception-related and comprehension-related tasks to work in harmony, as
they can effectively leverage the needed clues from the corresponding source
perception embeddings. Furthermore, to better evaluate the VTU task in
real-world scenarios, we establish a new and comprehensive table VQA benchmark,
ComTQA, featuring approximately 9,000 QA pairs. Extensive quantitative and
qualitative experiments on both table perception and comprehension tasks,
conducted across various public benchmarks, validate the effectiveness of our
TabPedia. The superior performance further confirms the feasibility of using
LLMs for understanding visual tables when all concepts work in synergy. The
benchmark ComTQA has been open-sourced at
https://huggingface.co/datasets/ByteDance/ComTQA. The source code and model
will be released later.
