Clustering-Based Ensembles as an Alternative to Stacking

Research output: Contribution to journalArticlepeer-review

22 Citations (Scopus)


One of the most popular techniques of generating classifier ensembles is known as stacking which is based on a meta-learning approach. In this paper we introduce an alternative method to stacking which is based on cluster analysis. Similar to stacking, instances from a validation set are initially classified by all base classifiers. The output of each classifier is subsequently considered as a new attribute of the instance. Following this, a validation set is divided into clusters according to the new attributes and a small subset of the original attributes of the instances. For each cluster we find its centroid and calculate its class label. The collection of centroids is considered as a meta-classifier. Experimental results show that the new method outperformed all benchmark methods, namely Majority Voting, Stacking J48, Stacking LR, AdaBoost J48 and Random Forest, in 12 out of 22 data sets. The proposed method has two advantageous properties: it is very robust to relatively small training sets and it can be applied in semi-supervised learning problems. We provide a theoretical investigation regarding the proposed method. This demonstrates that for the method to be successful, the base classifiers applied in the ensemble should have greater than 50% accuracy levels.
Original languageEnglish
Pages (from-to)2120-2137
Number of pages18
JournalIEEE Transactions on Knowledge and Data Engineering
Issue number9
Early online date20 Mar 2013
Publication statusPublished (in print/issue) - 30 Sept 2014


Dive into the research topics of 'Clustering-Based Ensembles as an Alternative to Stacking'. Together they form a unique fingerprint.

Cite this