  Endowing robots with the capability of assessing risk and making risk-aware
decisions is widely considered a key step toward ensuring safety for robots
operating under uncertainty. But, how should a robot quantify risk? A natural
and common approach is to consider the framework whereby costs are assigned to
stochastic outcomes - an assignment captured by a cost random variable.
Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping
from the cost random variable to a real number. Yet, the question of what
constitutes a "good" risk metric has received little attention within the
robotics community. The goal of this paper is to explore and partially address
this question by advocating axioms that risk metrics in robotics applications
should satisfy in order to be employed as rational assessments of risk. We
discuss general representation theorems that precisely characterize the class
of metrics that satisfy these axioms (referred to as distortion risk metrics),
and provide instantiations that can be used in applications. We further discuss
pitfalls of commonly used risk metrics in robotics, and discuss additional
properties that one must consider in sequential decision making tasks. Our hope
is that the ideas presented here will lead to a foundational framework for
quantifying risk (and hence safety) in robotics applications.
