  Evaluating forecasts is essential in order to understand and improve
forecasting and make forecasts useful to decision-makers. Much theoretical work
has been done on the development of proper scoring rules and other scoring
metrics that can help evaluate forecasts. In practice, however, conducting a
forecast evaluation and comparison of different forecasters remains
challenging. In this paper we introduce scoringutils, an R package that aims to
greatly facilitate this process. It is especially geared towards comparing
multiple forecasters, regardless of how forecasts were created, and visualising
results. The package is able to handle missing forecasts and is the first R
package to offer extensive support for forecasts represented through predictive
quantiles, a format used by several collaborative ensemble forecasting efforts.
The paper gives a short introduction to forecast evaluation, discusses the
metrics implemented in scoringutils and gives guidance on when they are
appropriate to use, and illustrates the application of the package using
example data of forecasts for COVID-19 cases and deaths submitted to the
European Forecast Hub between May and September 2021
