  In this paper, our goal is to generate synthetic data for heterogeneous
(mixed-type) tabular datasets with high machine learning utility (MLu). Given
that the MLu performance relies on accurately approximating the conditional
distributions, we focus on devising a synthetic data generation method based on
conditional distribution estimation. We propose a novel synthetic data
generation method, MaCoDE, by redefining the multi-class classification task of
Masked Language Modeling (MLM) as histogram-based non-parametric conditional
density estimation. Our proposed method enables estimating conditional
densities across arbitrary combinations of target and conditional variables.
Furthermore, we demonstrate that our proposed method bridges the theoretical
gap between distributional learning and MLM. To validate the effectiveness of
our proposed model, we conduct synthetic data generation experiments on 10
real-world datasets. Given the analogy between predicting masked input tokens
in MLM and missing data imputation, we also evaluate the performance of
multiple imputations on incomplete datasets with various missing data
mechanisms. Moreover, our proposed model offers the advantage of enabling
adjustments to data privacy levels without requiring re-training.
