  One of the main challenges in Interactive Information Retrieval (IIR)
evaluation is the development and application of re-usable tools that allow
researchers to analyze search behavior of real users in different environments
and different domains, but with comparable results. Furthermore, IIR recently
focuses more on the analysis of whole sessions, which includes all user
interactions that are carried out within a session but also across several
sessions by the same user. Some frameworks have already been proposed for the
evaluation of controlled experiments in IIR, but yet no framework is available
for interactive evaluation of search behavior from real-world information
retrieval (IR) systems with real users. In this paper we present a framework
for whole-session evaluation that can also utilize these uncontrolled data
sets. The logging component can easily be integrated into real-world IR systems
for generating and analyzing new log data. Furthermore, due to a supplementary
mapping it is also possible to analyze existing log data. For every IR system
different actions and filters can be defined. This allows system operators and
researchers to use the framework for the analysis of user search behavior in
their IR systems and to compare it with others. Using a graphical user
interface they have the possibility to interactively explore the data set from
a broad overview down to individual sessions.
