  In Interactive Information Retrieval (IIR) different services such as search
term suggestion can support users in their search process. The applicability
and performance of such services is either measured with different
user-centered studies (like usability tests or laboratory experiments) or, in
the context of IR, with their contribution to measures like precision and
recall. However, each evaluation methodology has its certain disadvantages. For
example, user-centered experiments are often costly and small-scaled; IR
experiments rely on relevance assessments and measure only relevance of
documents. In this work we operationalize the usefulness model of Cole et al.
(2009) on the level of system support to measure not only the local effect of
an IR service, but the impact it has on the whole search process. We therefore
use a log-based evaluation approach which models user interactions within
sessions with positive signals and apply it for the case of a search term
suggestion service. We found that the usage of the service significantly often
implicates the occurrence of positive signals during the following session
steps.
