  Language models (LMs) have been shown to memorize a great deal of factual
knowledge contained in their training data. But when an LM generates an
assertion, it is often difficult to determine where it learned this information
and whether it is true. In this paper, we propose the problem of fact tracing:
identifying which training examples taught an LM to generate a particular
factual assertion. Prior work on training data attribution (TDA) may offer
effective tools for identifying such examples, known as "proponents". We
present the first quantitative benchmark to evaluate this. We compare two
popular families of TDA methods -- gradient-based and embedding-based -- and
find that much headroom remains. For example, both methods have lower
proponent-retrieval precision than an information retrieval baseline (BM25)
that does not have access to the LM at all. We identify key challenges that may
be necessary for further improvement such as overcoming the problem of gradient
saturation, and also show how several nuanced implementation details of
existing neural TDA methods can significantly improve overall fact tracing
performance.
