  Machine learning models often incorporate vast amounts of data, raising
significant privacy concerns. Machine unlearning, the ability to remove the
influence of specific data points from a trained model, addresses these
concerns. This paper explores practical methods for implementing machine
unlearning, focusing on a first-epoch gradient-ascent approach.
  Key findings include: 1. Single vs. Multi-Epoch Unlearning: First-epoch
gradient unlearning is more effective than multi-epoch gradients. 2.
Layer-Based Unlearning: The embedding layer in GPT-2 is crucial for effective
unlearning. Gradients from the output layers (11 and 12) have no impact.
Efficient unlearning can be achieved using only the embedding layer, halving
space complexity. 3. Influence Functions & Scoring: Techniques like Hessian
Vector Product and the dot product of activations and tensors are used for
quantifying unlearning. 4. Gradient Ascent Considerations: Calibration is
necessary to avoid overexposing the model to specific data points during
unlearning, which could prematurely terminate the process. 5. Fuzzy Matching
vs. Iterative Unlearning: Fuzzy matching techniques shift the model to a new
optimum, while iterative unlearning provides a more complete modality.
  Our empirical evaluation confirms that first-epoch gradient ascent for
machine unlearning is more effective than whole-model gradient ascent. These
results highlight the potential of machine unlearning for enhancing data
privacy and compliance with regulations such as GDPR and CCPA. The study
underscores the importance of formal methods to comprehensively evaluate the
unlearning process.
