Quantifying consensus of rankings based on q-support patterns

Rankings, representing preferences over a set of candidates, are widely used in many applications, e.g., group decision making and information retrieval. Rankings may be obtained by different agents (humans or systems). It is often necessary to evaluate consensus of obtained rankings from multiple agents, as a measure of consensus provides insights into the rankings. Moreover, a consensus measure could provide a quantitative basis for comparing groups and for improving a ranking system. Existing studies on consensus measurement are insufficient, since they did not evaluate consensus among most rankings or consensus with respect to specific preference patterns. In this paper, a novel consensus quantifying approach, without the use of correlation or distance functions as in existing studies of consensus, is proposed based on the concept of q-support patterns, which represent the commonality embedded in a set of rankings. A pattern is regarded as a q-support pattern if it is included by at least q rankings in the ranking set. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.