  Tabular data is a common form of organizing data. Multiple models are
available to generate synthetic tabular datasets where observations are
independent, but few have the ability to produce relational datasets. Modeling
relational data is challenging as it requires modeling both a "parent" table
and its relationships across tables. We introduce REaLTabFormer (Realistic
Relational and Tabular Transformer), a tabular and relational synthetic data
generation model. It first creates a parent table using an autoregressive GPT-2
model, then generates the relational dataset conditioned on the parent table
using a sequence-to-sequence (Seq2Seq) model. We implement target masking to
prevent data copying and propose the $Q_{\delta}$ statistic and statistical
bootstrapping to detect overfitting. Experiments using real-world datasets show
that REaLTabFormer captures the relational structure better than a baseline
model. REaLTabFormer also achieves state-of-the-art results on prediction
tasks, "out-of-the-box", for large non-relational datasets without needing
fine-tuning.
