  Frontier artificial intelligence (AI) systems could pose increasing risks to
public safety and security. But what level of risk is acceptable? One
increasingly popular approach is to define capability thresholds, which
describe AI capabilities beyond which an AI system is deemed to pose too much
risk. A more direct approach is to define risk thresholds that simply state how
much risk would be too much. For instance, they might state that the likelihood
of cybercriminals using an AI system to cause X amount of economic damage must
not increase by more than Y percentage points. The main upside of risk
thresholds is that they are more principled than capability thresholds, but the
main downside is that they are more difficult to evaluate reliably. For this
reason, we currently recommend that companies (1) define risk thresholds to
provide a principled foundation for their decision-making, (2) use these risk
thresholds to help set capability thresholds, and then (3) primarily rely on
capability thresholds to make their decisions. Regulators should also explore
the area because, ultimately, they are the most legitimate actors to define
risk thresholds. If AI risk estimates become more reliable, risk thresholds
should arguably play an increasingly direct role in decision-making.
