  Data is the foundation of most science. Unfortunately, sharing data can be
obstructed by the risk of violating data privacy, impeding research in fields
like healthcare. Synthetic data is a potential solution. It aims to generate
data that has the same distribution as the original data, but that does not
disclose information about individuals. Membership Inference Attacks (MIAs) are
a common privacy attack, in which the attacker attempts to determine whether a
particular real sample was used for training of the model. Previous works that
propose MIAs against generative models either display low performance -- giving
the false impression that data is highly private -- or need to assume access to
internal generative model parameters -- a relatively low-risk scenario, as the
data publisher often only releases synthetic data, not the model. In this work
we argue for a realistic MIA setting that assumes the attacker has some
knowledge of the underlying data distribution. We propose DOMIAS, a
density-based MIA model that aims to infer membership by targeting local
overfitting of the generative model. Experimentally we show that DOMIAS is
significantly more successful at MIA than previous work, especially at
attacking uncommon samples. The latter is disconcerting since these samples may
correspond to underrepresented groups. We also demonstrate how DOMIAS' MIA
performance score provides an interpretable metric for privacy, giving data
publishers a new tool for achieving the desired privacy-utility trade-off in
their synthetic data.
