  We present two information leakage attacks that outperform previous work on
membership inference against generative models. The first attack allows
membership inference without assumptions on the type of the generative model.
Contrary to previous evaluation metrics for generative models, like Kernel
Density Estimation, it only considers samples of the model which are close to
training data records. The second attack specifically targets Variational
Autoencoders, achieving high membership inference accuracy. Furthermore,
previous work mostly considers membership inference adversaries who perform
single record membership inference. We argue for considering regulatory actors
who perform set membership inference to identify the use of specific datasets
for training. The attacks are evaluated on two generative model architectures,
Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs),
trained on standard image datasets. Our results show that the two attacks yield
success rates superior to previous work on most data sets while at the same
time having only very mild assumptions. We envision the two attacks in
combination with the membership inference attack type formalization as
especially useful. For example, to enforce data privacy standards and
automatically assessing model quality in machine learning as a service setups.
In practice, our work motivates the use of GANs since they prove less
vulnerable against information leakage attacks while producing detailed
samples.
