  Several membership inference (MI) attacks have been proposed to audit a
target DNN. Given a set of subjects, MI attacks tell which subjects the target
DNN has seen during training. This work focuses on the post-training MI attacks
emphasizing high confidence membership detection -- True Positive Rates (TPR)
at low False Positive Rates (FPR). Current works in this category -- likelihood
ratio attack (LiRA) and enhanced MI attack (EMIA) -- only perform well on
complex datasets (e.g., CIFAR-10 and Imagenet) where the target DNN overfits
its train set, but perform poorly on simpler datasets (0% TPR by both attacks
on Fashion-MNIST, 2% and 0% TPR respectively by LiRA and EMIA on MNIST at 1%
FPR). To address this, firstly, we unify current MI attacks by presenting a
framework divided into three stages -- preparation, indication and decision.
Secondly, we utilize the framework to propose two novel attacks: (1)
Adversarial Membership Inference Attack (AMIA) efficiently utilizes the
membership and the non-membership information of the subjects while
adversarially minimizing a novel loss function, achieving 6% TPR on both
Fashion-MNIST and MNIST datasets; and (2) Enhanced AMIA (E-AMIA) combines EMIA
and AMIA to achieve 8% and 4% TPRs on Fashion-MNIST and MNIST datasets
respectively, at 1% FPR. Thirdly, we introduce two novel augmented indicators
that positively leverage the loss information in the Gaussian neighborhood of a
subject. This improves TPR of all four attacks on average by 2.5% and 0.25%
respectively on Fashion-MNIST and MNIST datasets at 1% FPR. Finally, we propose
simple, yet novel, evaluation metric, the running TPR average (RTA) at a given
FPR, that better distinguishes different MI attacks in the low FPR region. We
also show that AMIA and E-AMIA are more transferable to the unknown DNNs (other
than the target DNN) and are more robust to DP-SGD training as compared to LiRA
and EMIA.
