  In public discussions of the quality of forecasts, attention typically
focuses on the predictive performance in cases of extreme events. However, the
restriction of conventional forecast evaluation methods to subsets of extreme
observations has unexpected and undesired effects, and is bound to discredit
skillful forecasts when the signal-to-noise ratio in the data generating
process is low. Conditioning on outcomes is incompatible with the theoretical
assumptions of established forecast evaluation methods, thereby confronting
forecasters with what we refer to as the forecaster's dilemma. For
probabilistic forecasts, proper weighted scoring rules have been proposed as
decision theoretically justifiable alternatives for forecast evaluation with an
emphasis on extreme events. Using theoretical arguments, simulation
experiments, and a real data study on probabilistic forecasts of U.S. inflation
and gross domestic product growth, we illustrate and discuss the forecaster's
dilemma along with potential remedies.
