  Prominent AI experts have suggested that companies developing high-risk AI
systems should be required to show that such systems are safe before they can
be developed or deployed. The goal of this paper is to expand on this idea and
explore its implications for risk management. We argue that entities developing
or deploying high-risk AI systems should be required to present evidence of
affirmative safety: a proactive case that their activities keep risks below
acceptable thresholds. We begin the paper by highlighting global security risks
from AI that have been acknowledged by AI experts and world governments. Next,
we briefly describe principles of risk management from other high-risk fields
(e.g., nuclear safety). Then, we propose a risk management approach for
advanced AI in which model developers must provide evidence that their
activities keep certain risks below regulator-set thresholds. As a first step
toward understanding what affirmative safety cases should include, we
illustrate how certain kinds of technical evidence and operational evidence can
support an affirmative safety case. In the technical section, we discuss
behavioral evidence (evidence about model outputs), cognitive evidence
(evidence about model internals), and developmental evidence (evidence about
the training process). In the operational section, we offer examples of
organizational practices that could contribute to affirmative safety cases:
information security practices, safety culture, and emergency response
capacity. Finally, we briefly compare our approach to the NIST AI Risk
Management Framework. Overall, we hope our work contributes to ongoing
discussions about national and global security risks posed by AI and regulatory
approaches to address these risks.
