<p>The standard approach to computing the correlation between two total orders is <a href="http://en.wikipedia.org/wiki/Kendall_tau_rank_correlation_coefficient" rel="nofollow noreferrer">Kendall's $\tau$</a>, defined as the difference between the number of agreeing pairs and disagreeing pairs (in rank) divided by $\binom{n}{2}$. </p>

<p>If you only have partial orders, then <a href="https://mathoverflow.net/questions/15955/defining-average-rank-when-not-every-ranking-covers-the-whole-set/15958#15958">things get a little trickier</a>. </p>

CC BY-SA 2.5
2011-02-23T03:39:36.953Z
Many thanks for the answer and pointers! "Rank correlation" and Google was what I needed... :)
badroit
CC BY-SA 2.5
2011-02-23T03:30:33.317Z
2011-02-23T03:30:33.317Z
2017-04-13T12:57:54.777Z
Suresh Venkat
<p>An easy question perhaps?</p>

<p>Taking a (fictional) concrete example, let's say I have two ranking methods for HTML documents: PageRank and HITS. I derive an ordered list over the same set of documents according to their rank values (there can be ties) for both ranking methods. </p>

<p>Now, I want to quantify how much the two orderings correlate. I'm not interested in the absolute values of the ranking measures, but I'm looking for measures of correlation between the orderings.</p>

<p>I can think of some intuitive correlation measures one could apply like "average distance" or such: </p>

<p>${\displaystyle\frac{\sum_{doc_i\in Docs}\mathrm{abs}(pos_{rank1}(doc_i) - pos_{rank2}(doc_i))}{|Docs|^2}}$</p>

<p>but I'm wondering if anyone knows of any standard techniques I could check out? </p>

<p>In particular, I would prefer a symmetric measure which is easy to intuit and whose results are tangible/"portable". I've a number of different orderings which I'll be comparing pair-wise, but the set of $Docs$ remains the same.</p>

<p>Many thanks in advance! </p>

<p>(having difficulty tagging... please feel free to edit tags)</p>

<co.combinatorics><st.statistics><order-theory>
Measures of "correlation" between two orderings
badroit
# Measures of "correlation" between two orderings

(Asked by: username_0 on 2011-02-23)

An easy question perhaps?

Taking a (fictional) concrete example, let's say I have two ranking methods for HTML documents: PageRank and HITS. I derive an ordered list over the same set of documents according to their rank values (there can be ties) for both ranking methods.

Now, I want to quantify how much the two orderings correlate. I'm not interested in the absolute values of the ranking measures, but I'm looking for measures of correlation between the orderings.

I can think of some intuitive correlation measures one could apply like "average distance" or such:

${\displaystyle\frac{\sum\_{doc\_i\in Docs}\mathrm{abs}(pos\_{rank1}(doc\_i) - pos\_{rank2}(doc\_i))}{|Docs|^2}}$

but I'm wondering if anyone knows of any standard techniques I could check out?

In particular, I would prefer a symmetric measure which is easy to intuit and whose results are tangible/"portable". I've a number of different orderings which I'll be comparing pair-wise, but the set of $Docs$ remains the same.

Many thanks in advance!

(having difficulty tagging... please feel free to edit tags)

**Tags:** co.combinatorics, st.statistics, order-theory


---

## 1 answers

### Answer by username_1 on 2011-02-23. Score: 8 | Accepted answer

The standard approach to computing the correlation between two total orders is [Kendall's $\tau$](http://en.wikipedia.org/wiki/Kendall_tau_rank_correlation_coefficient), defined as the difference between the number of agreeing pairs and disagreeing pairs (in rank) divided by $\binom{n}{2}$.

If you only have partial orders, then [things get a little trickier](https://mathoverflow.net/questions/15955/defining-average-rank-when-not-every-ranking-covers-the-whole-set/15958#15958).

> **Comments:**
> **username_0**: Many thanks for the answer and pointers! "Rank correlation" and Google was what I needed... :)  

---

