  Despite the retrieval effectiveness of queries being mutually independent of
one another, the evaluation of query performance prediction (QPP) systems has
been carried out by measuring rank correlation over an entire set of queries.
Such a listwise approach has a number of disadvantages, notably that it does
not support the common requirement of assessing QPP for individual queries. In
this paper, we propose a pointwise QPP framework that allows us to evaluate the
quality of a QPP system for individual queries by measuring the deviations
between each prediction versus the corresponding true value, and then
aggregating the results over a set of queries. Our experiments demonstrate that
this new approach leads to smaller variances in QPP evaluations across a range
of different target metrics and retrieval models.
