  This paper describes problems with the current way we compare the diversity
of different recommendation lists in offline experiments. We illustrate the
problems with a case study. We propose the Sudden Death score as a new and
better way of making these comparisons.
