  Forecast evaluations aim to choose an accurate forecast for making decisions
by using loss functions. However, different loss functions often generate
different ranking results for forecasts, which complicates the task of
comparisons. In this paper, we develop statistical tests for comparing
performances of forecasting expectiles and quantiles of a random variable under
consistent loss functions. The test statistics are constructed with the
extremal consistent loss functions of Ehm et.al. (2016). The null hypothesis of
the tests is that a benchmark forecast at least performs equally well as a
competing one under all extremal consistent loss functions. It can be shown
that if such a null holds, the benchmark will also perform at least equally
well as the competitor under all consistent loss functions. Thus under the
null, when different consistent loss functions are used, the result that the
competitor does not outperform the benchmark will not be altered. We establish
asymptotic properties of the proposed test statistics and propose to use the
re-centered bootstrap to construct their empirical distributions. Through
simulations, we show the proposed test statistics perform reasonably well. We
then apply the proposed method on (1) re-examining abilities of some often-used
predictors on forecasting risk premium of the S&P500 index; (2) comparing
performances of experts' forecasts on annual growth of U.S. real gross domestic
product; (3) evaluating performances of estimated daily value at risk of the
S&P500 index.
