Overall rank from multiple ranked lists

I’ve looked through a lot of literature available online, including this forum without any luck and hoping someone can help a statistical issue I currently face:

I have 5 lists of of ranked data, each containing 10 items ranked from position 1 (best) to position 10 (worst). For sake of context, the 10 items in each lists are the same, but in different ranked orders as the technique used to decide their rank is different.

Example data:

            List 1      List 2      List 3     ... etc
Item 1     Ranked 1    Ranked 2    Ranked 1     
Item 2     Ranked 3    Ranked 1    Ranked 2
Item 3     Ranked 2    Ranked 3    Ranked 3
... etc

I am looking for a way to interpret and analyse the above data so that I get a final result showing the overall rank of each item based on each test and its position, e.g.

Result
Rank 1 = Item 1
Rank 2 = Item 3
Rank 3 = Item 4
... etc

So far I have attempted to interpret this information from performing Pearson’s Correlation, Spearman’s Correlation, Kendall Tau’s B, and Friedman tests. I have found however, that these results have generally paired my lists (i.e. compared list 1 to list 2, then list 1 to list 3 .. etc), or have produced results such as Chi-Square, P-Values etc about the overall data.

Does anyone know how I can interpret this data in a statistically sound method (at a post graduate / PhD applicable level) so that I can understand the overall ranks signalling the importance of each item in the list across the 5 tests please? Or, if there is another type of technique or statistical test I can look into I would appreciate any hints or guidance.

(It maybe also worth noting, I have also performed the simpler mathematical techniques such as sums, averaging, minimum – maximum tests etc, but do not feel these are statistically important enough at this level).

Any help or advice would be greatly appreciated, thank you for your time.

Answer

I am not sure why you were looking at correlations and similar measures. There doesn’t seem to be anything to correlate.

Instead, there are a number of options, none really better than the other, but depending on what you want:

Take the average rank and then rank the averages (but this treats the data as interval)

Take the median rank and then rank the medians (but this may result in ties)

Take the number of 1st place votes each item got, and rank them based on this

Take the number of last place votes and rank them (inversely, obviously) based on that.

Create some weighted combination of ranks, depending on what you think reasonable.

Attribution
Source : Link , Question Author : Liam , Answer Author : Peter Flom

Leave a Comment