  The Fractions Skill Score (FSS) is a widely used metric for assessing
forecast skill, with applications ranging from precipitation to volcanic ash
forecasts. By evaluating the fraction of grid squares exceeding a threshold in
a neighbourhood, the intuition is that it can avoid the pitfalls of pixel-wise
comparisons and identify length scales at which a forecast has skill. The FSS
is typically interpreted relative to a `useful' criterion, which provides a
measure of whether a forecast is considered skilful enough. However, this
useful criterion is problematic, as there can be considerable skill for
forecasts that do not meet it, and it is not derived in a way that provides
obvious meaning. We therefore provide a new derivation of a useful criterion,
as the score that a random forecast achieves, which provides a more robust and
meaningful way to assess the FSS. Through illustrative examples we show that
this new criterion considerably changes the length scales at which a forecast
would be regarded as skilful, and reveals subtleties in how the FSS should be
interpreted.
