  With the passage of the Right to Be Forgotten (RTBF) regulations and the
scaling up of language model training datasets, research on model unlearning in
large language models (LLMs) has become more crucial. Before the era of LLMs,
machine unlearning research focused mainly on classification tasks in models
with small parameters. In these tasks, the content to be forgotten or retained
is clear and straightforward. However, as parameter sizes have grown and tasks
have become more complex, balancing forget quality and model utility has become
more challenging, especially in scenarios involving personal data instead of
classification results. Existing methods based on gradient ascent and its
variants often struggle with this balance, leading to unintended information
loss or partial forgetting. To address this challenge, we propose RKLD, a novel
\textbf{R}everse \textbf{KL}-Divergence-based Knowledge \textbf{D}istillation
unlearning algorithm for LLMs targeting the unlearning of personal information.
Through RKLD, we achieve significant forget quality and effectively maintain
the model utility in our experiments.
