  The widespread use of generative AI systems is coupled with significant
ethical and social challenges. As a result, policymakers, academic researchers,
and social advocacy groups have all called for such systems to be audited.
However, existing auditing procedures fail to address the governance challenges
posed by generative AI systems, which display emergent capabilities and are
adaptable to a wide range of downstream tasks. In this chapter, we address that
gap by outlining a novel blueprint for how to audit such systems. Specifically,
we propose a three-layered approach, whereby governance audits (of technology
providers that design and disseminate generative AI systems), model audits (of
generative AI systems after pre-training but prior to their release), and
application audits (of applications based on top of generative AI systems)
complement and inform each other. We show how audits on these three levels,
when conducted in a structured and coordinated manner, can be a feasible and
effective mechanism for identifying and managing some of the ethical and social
risks posed by generative AI systems. That said, it is important to remain
realistic about what auditing can reasonably be expected to achieve. For this
reason, the chapter also discusses the limitations not only of our
three-layered approach but also of the prospect of auditing generative AI
systems at all. Ultimately, this chapter seeks to expand the methodological
toolkit available to technology providers and policymakers who wish to analyse
and evaluate generative AI systems from technical, ethical, and legal
perspectives.
