  This paper investigates Who's Harry Potter (WHP), a pioneering yet
insufficiently understood method for LLM unlearning. We explore it in two
steps. First, we introduce a new task of LLM targeted unlearning, where given
an unlearning target (e.g., a person) and some unlearning documents, we aim to
unlearn only the information about the target, rather than everything in the
unlearning documents. We further argue that a successful unlearning should
satisfy criteria such as not outputting gibberish, not fabricating facts about
the unlearning target, and not releasing factual information under jailbreak
attacks. Second, we construct a causal intervention framework for targeted
unlearning, where the knowledge of the unlearning target is modeled as a
confounder between LLM input and output, and the unlearning process as a
deconfounding process. This framework justifies and extends WHP, deriving a
simple unlearning algorithm that includes WHP as a special case. Experiments on
existing and new datasets show that our approach, without explicitly optimizing
for the aforementioned criteria, achieves competitive performance in all of
them. Our code is available at
https://github.com/UCSB-NLP-Chang/causal_unlearn.git.
