  Large language models (LLMs) have shown impressive results while requiring
little or no direct supervision. Further, there is mounting evidence that LLMs
may have potential in information-seeking scenarios. We believe the ability of
an LLM to attribute the text that it generates is likely to be crucial in this
setting. We formulate and study Attributed QA as a key first step in the
development of attributed LLMs. We propose a reproducible evaluation framework
for the task and benchmark a broad set of architectures. We take human
annotations as a gold standard and show that a correlated automatic metric is
suitable for development. Our experimental work gives concrete answers to two
key questions (How to measure attribution?, and How well do current
state-of-the-art methods perform on attribution?), and give some hints as to
how to address a third (How to build LLMs with attribution?).
