  Kernel mean embedding is a useful tool to represent and compare probability
measures. Despite its usefulness, kernel mean embedding considers
infinite-dimensional features, which are challenging to handle in the context
of differentially private data generation. A recent work proposes to
approximate the kernel mean embedding of data distribution using
finite-dimensional random features, which yields analytically tractable
sensitivity. However, the number of required random features is excessively
high, often ten thousand to a hundred thousand, which worsens the
privacy-accuracy trade-off. To improve the trade-off, we propose to replace
random features with Hermite polynomial features. Unlike the random features,
the Hermite polynomial features are ordered, where the features at the low
orders contain more information on the distribution than those at the high
orders. Hence, a relatively low order of Hermite polynomial features can more
accurately approximate the mean embedding of the data distribution compared to
a significantly higher number of random features. As demonstrated on several
tabular and image datasets, Hermite polynomial features seem better suited for
private data generation than random Fourier features.
