  Relevance and diversity are both crucial criteria for an effective search
system. In this paper, we propose a unified learning framework for
simultaneously optimizing both relevance and diversity. Specifically, the
problem is formalized as a structural learning framework optimizing
Diversity-Correlated Evaluation Measures (DCEM), such as ERR-IA, a-NDCG and
NRBP. Within this framework, the discriminant function is defined to be a
bi-criteria objective maximizing the sum of the relevance scores and
dissimilarities (or diversity) among the documents. Relevance and diversity
features are utilized to define the relevance scores and dissimilarities,
respectively. Compared with traditional methods, the advantages of our approach
lie in that: (1) Directly optimizing DCEM as the loss function is more
fundamental for the task; (2) Our framework does not rely on explicit diversity
information such as subtopics, thus is more adaptive to real application; (3)
The representation of diversity as the feature-based scoring function is more
flexible to incorporate rich diversity-based features into the learning
framework. Extensive experiments on the public TREC datasets show that our
approach significantly outperforms state-of-the-art diversification approaches,
which validate the above advantages.
