Quantifying consensus of rankings based on q-support patterns

Zhengui Xue, Zhiwei Lin, Hui Wang, Sally McClean

Research output: Contribution to journalArticle

Abstract

Rankings, representing preferences over a set of candidates, are widely used in many applications, e.g., group decision making and information retrieval. Rankings may be obtained by different agents (humans or systems). It is often necessary to evaluate consensus of obtained rankings from multiple agents, as a measure of consensus provides insights into the rankings. Moreover, a consensus measure could provide a quantitative basis for comparing groups and for improving a ranking system. Existing studies on consensus measurement are insufficient, since they did not evaluate consensus among most rankings or consensus with respect to specific preference patterns. In this paper, a novel consensus quantifying approach, without the use of correlation or distance functions as in existing studies of consensus, is proposed based on the concept of q-support patterns, which represent the commonality embedded in a set of rankings. A pattern is regarded as a q-support pattern if it is included by at least q rankings in the ranking set. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.
LanguageEnglish
Pages396-412
Number of pages17
JournalInformation Sciences
Volume518
Early online date28 Dec 2019
DOIs
Publication statusE-pub ahead of print - 28 Dec 2019

Fingerprint

Information retrieval
Ranking
Decision making
Group Decision Making
Evaluate
Distance Function
Information Retrieval
Outlier
Correlation Function
Experimental Study
Necessary

Cite this

@article{8551957e370a45ee8674d84cee8508d7,
title = "Quantifying consensus of rankings based on q-support patterns",
abstract = "Rankings, representing preferences over a set of candidates, are widely used in many applications, e.g., group decision making and information retrieval. Rankings may be obtained by different agents (humans or systems). It is often necessary to evaluate consensus of obtained rankings from multiple agents, as a measure of consensus provides insights into the rankings. Moreover, a consensus measure could provide a quantitative basis for comparing groups and for improving a ranking system. Existing studies on consensus measurement are insufficient, since they did not evaluate consensus among most rankings or consensus with respect to specific preference patterns. In this paper, a novel consensus quantifying approach, without the use of correlation or distance functions as in existing studies of consensus, is proposed based on the concept of q-support patterns, which represent the commonality embedded in a set of rankings. A pattern is regarded as a q-support pattern if it is included by at least q rankings in the ranking set. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.",
author = "Zhengui Xue and Zhiwei Lin and Hui Wang and Sally McClean",
year = "2019",
month = "12",
day = "28",
doi = "10.1016/j.ins.2019.12.070",
language = "English",
volume = "518",
pages = "396--412",
journal = "Information Sciences",
issn = "0020-0255",
publisher = "Elsevier",

}

Quantifying consensus of rankings based on q-support patterns. / Xue, Zhengui; Lin, Zhiwei; Wang, Hui; McClean, Sally.

In: Information Sciences, Vol. 518, 31.05.2020, p. 396-412.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Quantifying consensus of rankings based on q-support patterns

AU - Xue, Zhengui

AU - Lin, Zhiwei

AU - Wang, Hui

AU - McClean, Sally

PY - 2019/12/28

Y1 - 2019/12/28

N2 - Rankings, representing preferences over a set of candidates, are widely used in many applications, e.g., group decision making and information retrieval. Rankings may be obtained by different agents (humans or systems). It is often necessary to evaluate consensus of obtained rankings from multiple agents, as a measure of consensus provides insights into the rankings. Moreover, a consensus measure could provide a quantitative basis for comparing groups and for improving a ranking system. Existing studies on consensus measurement are insufficient, since they did not evaluate consensus among most rankings or consensus with respect to specific preference patterns. In this paper, a novel consensus quantifying approach, without the use of correlation or distance functions as in existing studies of consensus, is proposed based on the concept of q-support patterns, which represent the commonality embedded in a set of rankings. A pattern is regarded as a q-support pattern if it is included by at least q rankings in the ranking set. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.

AB - Rankings, representing preferences over a set of candidates, are widely used in many applications, e.g., group decision making and information retrieval. Rankings may be obtained by different agents (humans or systems). It is often necessary to evaluate consensus of obtained rankings from multiple agents, as a measure of consensus provides insights into the rankings. Moreover, a consensus measure could provide a quantitative basis for comparing groups and for improving a ranking system. Existing studies on consensus measurement are insufficient, since they did not evaluate consensus among most rankings or consensus with respect to specific preference patterns. In this paper, a novel consensus quantifying approach, without the use of correlation or distance functions as in existing studies of consensus, is proposed based on the concept of q-support patterns, which represent the commonality embedded in a set of rankings. A pattern is regarded as a q-support pattern if it is included by at least q rankings in the ranking set. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.

U2 - 10.1016/j.ins.2019.12.070

DO - 10.1016/j.ins.2019.12.070

M3 - Article

VL - 518

SP - 396

EP - 412

JO - Information Sciences

T2 - Information Sciences

JF - Information Sciences

SN - 0020-0255

ER -