A Study of Evaluation Metrics for Recommender Algorithms

Jennifer Redpath; CM Shapcott; SI McClean; Liming Chen

A Study of Evaluation Metrics for Recommender Algorithms

Jennifer Redpath
, CM Shapcott
, SI McClean
, Liming Chen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

There are inherent problems with evaluating the accuracy of recommender systems. Commonly-used metrics for recommender systems depend on the number of recommendations produced and the number of hidden items withheld, making it difficult to directly compare one system with another. In this paper we compare recommender algorithms using two datasets; the standard MovieLens set and an e-commerce dataset that has implicit ratings based on browsing behaviour. We introduce a measure that aids in the comparison and show how to compare results with baseline predictions based on random recommendation selections.

Original language	English
Title of host publication	Unknown Host Publication
Publisher	AICS
Number of pages	10
Publication status	Published (in print/issue) - 25 Aug 2008
Event	The 19th Irish Conference on Artificial Intelligence and Cognitive Science - Duration: 25 Aug 2008 → …

Conference

Conference	The 19th Irish Conference on Artificial Intelligence and Cognitive Science
Period	25/08/08 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Cite this

@inproceedings{a9a18508e054488485c38a4fbc1ddb05,

title = "A Study of Evaluation Metrics for Recommender Algorithms",

abstract = "There are inherent problems with evaluating the accuracy of recommender systems. Commonly-used metrics for recommender systems depend on the number of recommendations produced and the number of hidden items withheld, making it difficult to directly compare one system with another. In this paper we compare recommender algorithms using two datasets; the standard MovieLens set and an e-commerce dataset that has implicit ratings based on browsing behaviour. We introduce a measure that aids in the comparison and show how to compare results with baseline predictions based on random recommendation selections.",

author = "Jennifer Redpath and CM Shapcott and SI McClean and Liming Chen",

year = "2008",

month = aug,

day = "25",

language = "English",

booktitle = "Unknown Host Publication",

publisher = "AICS",

address = "Ireland",

note = "The 19th Irish Conference on Artificial Intelligence and Cognitive Science ; Conference date: 25-08-2008",

}

TY - GEN

T1 - A Study of Evaluation Metrics for Recommender Algorithms

AU - Redpath, Jennifer

AU - Shapcott, CM

AU - McClean, SI

AU - Chen, Liming

PY - 2008/8/25

Y1 - 2008/8/25

N2 - There are inherent problems with evaluating the accuracy of recommender systems. Commonly-used metrics for recommender systems depend on the number of recommendations produced and the number of hidden items withheld, making it difficult to directly compare one system with another. In this paper we compare recommender algorithms using two datasets; the standard MovieLens set and an e-commerce dataset that has implicit ratings based on browsing behaviour. We introduce a measure that aids in the comparison and show how to compare results with baseline predictions based on random recommendation selections.

AB - There are inherent problems with evaluating the accuracy of recommender systems. Commonly-used metrics for recommender systems depend on the number of recommendations produced and the number of hidden items withheld, making it difficult to directly compare one system with another. In this paper we compare recommender algorithms using two datasets; the standard MovieLens set and an e-commerce dataset that has implicit ratings based on browsing behaviour. We introduce a measure that aids in the comparison and show how to compare results with baseline predictions based on random recommendation selections.

M3 - Conference contribution

BT - Unknown Host Publication

PB - AICS

T2 - The 19th Irish Conference on Artificial Intelligence and Cognitive Science

Y2 - 25 August 2008

ER -

A Study of Evaluation Metrics for Recommender Algorithms

Abstract

Conference

UN SDGs

Fingerprint

Cite this