  Machine learning classification methods usually assume that all possible
classes are sufficiently present within the training set. Due to their inherent
rarities, extreme events are always under-represented and classifiers tailored
for predicting extremes need to be carefully designed to handle this
under-representation. In this paper, we address the question of how to assess
and compare classifiers with respect to their capacity to capture extreme
occurrences. This is also related to the topic of scoring rules used in
forecasting literature. In this context, we propose and study a risk function
adapted to extremal classifiers. The inferential properties of our empirical
risk estimator are derived under the framework of multivariate regular
variation and hidden regular variation. A simulation study compares different
classifiers and indicates their performance with respect to our risk function.
To conclude, we apply our framework to the analysis of extreme river discharges
in the Danube river basin. The application compares different predictive
algorithms and test their capacity at forecasting river discharges from other
river stations.
