  To comply with AI and data regulations, the need to forget private or
copyrighted information from trained machine learning models is increasingly
important. The key challenge in unlearning is forgetting the necessary data in
a timely manner, while preserving model performance. In this work, we address
the zero-shot unlearning scenario, whereby an unlearning algorithm must be able
to remove data given only a trained model and the data to be forgotten. We
explore unlearning from an information theoretic perspective, connecting the
influence of a sample to the information gain a model receives by observing it.
From this, we derive a simple but principled zero-shot unlearning method based
on the geometry of the model. Our approach takes the form of minimising the
gradient of a learned function with respect to a small neighbourhood around a
target forget point. This induces a smoothing effect, causing forgetting by
moving the boundary of the classifier. We explore the intuition behind why this
approach can jointly unlearn forget samples while preserving general model
performance through a series of low-dimensional experiments. We perform
extensive empirical evaluation of our method over a range of contemporary
benchmarks, verifying that our method is competitive with state-of-the-art
performance under the strict constraints of zero-shot unlearning.
