An approach for measuring semantic similarity between words using multiple information sources

Yuhua Li; Zuhair Bandar; David McLean

An approach for measuring semantic similarity between words using multiple information sources

Yuhua Li, Zuhair Bandar, David McLean

Research output: Contribution to journal › Article › peer-review

Abstract

Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.

Original language	English
Pages (from-to)	871-882
Journal	IEEE Transactions on Knowledge and Data Engineering
Volume	15
Issue number	4
Publication status	Published (in print/issue) - Jul 2003

Bibliographical note

This paper rigorously investigates the contributions of different information sources to similarity between words. It presents word similarity measures by nonlinearly combining structural semantic information from lexical taxonomy and information content from corpus. Our approach outperforms previously published measures: best published correlation against the benchmark set of word pairs of Rubenstein-Goodenough's human similarity ratings has been 0.8484, whilst ours is 0.8914. The paper has been cited over 70 times (SCI) and 300 times (Google Scholar) as of Jan 2010, selected as advanced reading material in CIS526 Machine Learning, Temple University, Philadelphia, and adopted by other researchers in real system developments, e.g., Bibster - a semantics-based bibliographic peer-to-peer system.

Access to Document

http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/trans/tk/&toc=comp/trans/tk/2003/04/k4toc.xml&DOI=10.1109/TKDE.2003.1209005

Cite this

@article{74ec2f2034f146d6b6229a1bf29b61af,

title = "An approach for measuring semantic similarity between words using multiple information sources",

abstract = "Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.",

author = "Yuhua Li and Zuhair Bandar and David McLean",

note = "This paper rigorously investigates the contributions of different information sources to similarity between words. It presents word similarity measures by nonlinearly combining structural semantic information from lexical taxonomy and information content from corpus. Our approach outperforms previously published measures: best published correlation against the benchmark set of word pairs of Rubenstein-Goodenough's human similarity ratings has been 0.8484, whilst ours is 0.8914. The paper has been cited over 70 times (SCI) and 300 times (Google Scholar) as of Jan 2010, selected as advanced reading material in CIS526 Machine Learning, Temple University, Philadelphia, and adopted by other researchers in real system developments, e.g., Bibster - a semantics-based bibliographic peer-to-peer system.",

year = "2003",

month = jul,

language = "English",

volume = "15",

pages = "871--882",

journal = "IEEE Transactions on Knowledge and Data Engineering",

publisher = "IEEE",

number = "4",

}

TY - JOUR

T1 - An approach for measuring semantic similarity between words using multiple information sources

AU - Li, Yuhua

AU - Bandar, Zuhair

AU - McLean, David

N1 - This paper rigorously investigates the contributions of different information sources to similarity between words. It presents word similarity measures by nonlinearly combining structural semantic information from lexical taxonomy and information content from corpus. Our approach outperforms previously published measures: best published correlation against the benchmark set of word pairs of Rubenstein-Goodenough's human similarity ratings has been 0.8484, whilst ours is 0.8914. The paper has been cited over 70 times (SCI) and 300 times (Google Scholar) as of Jan 2010, selected as advanced reading material in CIS526 Machine Learning, Temple University, Philadelphia, and adopted by other researchers in real system developments, e.g., Bibster - a semantics-based bibliographic peer-to-peer system.

PY - 2003/7

Y1 - 2003/7

N2 - Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.

AB - Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.

UR - http://computer.org/tkde/

M3 - Article

VL - 15

SP - 871

EP - 882

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

IS - 4

ER -

An approach for measuring semantic similarity between words using multiple information sources

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this