  Large language models (LLMs) sometimes exhibit dangerous unintended
behaviors. Finding and fixing these is challenging because the attack surface
is massive -- it is not tractable to exhaustively search for all possible
inputs that may elicit such behavior. One specific and particularly challenging
case is that if data-poisoning-injected trojans, since there is no way to know
what they are to search for them. To our knowledge, there is no generally
applicable method to unlearn unknown trojans injected during pre-training. This
work seeks to provide a general purpose recipe (filters) and a specific
implementation (LoRA) filters that work in practice on small to medium sized
models. The focus is primarily empirical, though some perplexing behavior opens
the door to the fundamental question of how LLMs store and process information.
Not unexpectedly, we find that our filters work best on the residual stream and
the latest layers.
