  We explore generating factual and accurate tables from the parametric
knowledge of large language models (LLMs). While LLMs have demonstrated
impressive capabilities in recreating knowledge bases and generating free-form
text, we focus on generating structured tabular data, which is crucial in
domains like finance and healthcare. We examine the table generation abilities
of four state-of-the-art LLMs: GPT-3.5, GPT-4, Llama2-13B, and Llama2-70B,
using three prompting methods for table generation: (a) full-table, (b)
row-by-row; (c) cell-by-cell. For evaluation, we introduce a novel benchmark,
WikiTabGen which contains 100 curated Wikipedia tables. Tables are further
processed to ensure their factual correctness and manually annotated with short
natural language descriptions. Our findings reveal that table generation
remains a challenge, with GPT-4 reaching the highest accuracy at 19.6%. Our
detailed analysis sheds light on how various table properties, such as size,
table popularity, and numerical content, influence generation performance. This
work highlights the unique challenges in LLM-based table generation and
provides a solid evaluation framework for future research. Our code, prompts
and data are all publicly available:
https://github.com/analysis-bots/WikiTabGen
