  Recent breakthroughs in large language modeling have facilitated rigorous
exploration of their application in diverse tasks related to tabular data
modeling, such as prediction, tabular data synthesis, question answering, and
table understanding. Each task presents unique challenges and opportunities.
However, there is currently a lack of comprehensive review that summarizes and
compares the key techniques, metrics, datasets, models, and optimization
approaches in this research domain. This survey aims to address this gap by
consolidating recent progress in these areas, offering a thorough survey and
taxonomy of the datasets, metrics, and methodologies utilized. It identifies
strengths, limitations, unexplored territories, and gaps in the existing
literature, while providing some insights for future research directions in
this vital and rapidly evolving field. It also provides relevant code and
datasets references. Through this comprehensive review, we hope to provide
interested readers with pertinent references and insightful perspectives,
empowering them with the necessary tools and knowledge to effectively navigate
and address the prevailing challenges in the field.
