  Pretrained language models memorize vast amounts of information, including
private and copyrighted data, raising significant safety concerns. Retraining
these models after excluding sensitive data is prohibitively expensive, making
machine unlearning a viable, cost-effective alternative. Previous research has
focused on machine unlearning for monolingual models, but we find that
unlearning in one language does not necessarily transfer to others. This
vulnerability makes models susceptible to low-resource language attacks, where
sensitive information remains accessible in less dominant languages. This paper
presents a pioneering approach to machine unlearning for multilingual language
models, selectively erasing information across different languages while
maintaining overall performance. Specifically, our method employs an adaptive
unlearning scheme that assigns language-dependent weights to address different
language performances of multilingual language models. Empirical results
demonstrate the effectiveness of our framework compared to existing unlearning
baselines, setting a new standard for secure and adaptable multilingual
language models.
