  Verifying probabilistic forecasts for extreme events is a highly active
research area because popular media and public opinions are naturally focused
on extreme events, and biased conclusions are readily made. In this context,
classical verification methods tailored for extreme events, such as thresholded
and weighted scoring rules, have undesirable properties that cannot be
mitigated, and the well-known continuous ranked probability score (CRPS) is no
exception.
  In this paper, we define a formal framework for assessing the behavior of
forecast evaluation procedures with respect to extreme events, which we use to
demonstrate that assessment based on the expectation of a proper score is not
suitable for extremes. Alternatively, we propose studying the properties of the
CRPS as a random variable by using extreme value theory to address extreme
event verification. An index is introduced to compare calibrated forecasts,
which summarizes the ability of probabilistic forecasts for predicting
extremes. The strengths and limitations of this method are discussed using both
theoretical arguments and simulations.
