  Relevance is generally understood as a multi-level and multi-dimensional
relationship between an information need and an information object. However,
traditional IR evaluation metrics naively assume mono-dimensionality. We ask:
How to deal with multidimensional and graded relevance assessments in IR
evaluation? Moreover, search result evaluation metrics neglect document
overlaps and naively assume gains piling up as the searcher examines the ranked
list into greater length. Consequently, we examine: How to deal with document
overlap in IR evaluation? The usability of a document for a person-in-need also
depends on document usability attributes beyond relevance. Therefore, we ask:
How to deal with usability attributes, and how to combine this with
multidimensional relevance assessments in IR evaluation? Finally, we ask how to
define a formal model, which deals with multidimensional graded relevance
assessments, document overlaps, and document usability attributes in a coherent
framework serving IR evaluation?
