  We consider membership inference attacks, one of the main privacy issues in
machine learning. These recently developed attacks have been proven successful
in determining, with confidence better than a random guess, whether a given
sample belongs to the dataset on which the attacked machine learning model was
trained. Several approaches have been developed to mitigate this privacy
leakage but the tradeoff performance implications of these defensive mechanisms
(i.e., accuracy and utility of the defended machine learning model) are not
well studied yet. We propose a novel approach of privacy leakage avoidance with
switching ensembles (PASE), which both protects against current membership
inference attacks and does that with very small accuracy penalty, while
requiring acceptable increase in training and inference time. We test our PASE
method, along with the the current state-of-the-art PATE approach, on three
calibration image datasets and analyze their tradeoffs.
