  With the continued advancement and widespread adoption of machine learning
(ML) models across various domains, ensuring user privacy and data security has
become a paramount concern. In compliance with data privacy regulations, such
as GDPR, a secure machine learning framework should not only grant users the
right to request the removal of their contributed data used for model training
but also facilitates the elimination of sensitive data fingerprints within
machine learning models to mitigate potential attack - a process referred to as
machine unlearning. In this study, we present a novel unlearning mechanism
designed to effectively remove the impact of specific data samples from a
neural network while considering the performance of the unlearned model on the
primary task. In achieving this goal, we crafted a novel loss function tailored
to eliminate privacy-sensitive information from weights and activation values
of the target model by combining target classification loss and membership
inference loss. Our adaptable framework can easily incorporate various privacy
leakage approximation mechanisms to guide the unlearning process. We provide
empirical evidence of the effectiveness of our unlearning approach with a
theoretical upper-bound analysis through a membership inference mechanism as a
proof of concept. Our results showcase the superior performance of our approach
in terms of unlearning efficacy and latency as well as the fidelity of the
primary task, across four datasets and four deep learning architectures.
