A Dynamic Ensemble Learning Algorithm for Neural Networks

N Siddique, Kazi Md. Rokibul Alam, Hojjat Adeli

Research output: Contribution to journalArticle

Abstract

This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.
LanguageEnglish
Number of pages16
JournalNeural Computing and Applications
DOIs
Publication statusPublished - 28 Jul 2019

Fingerprint

Learning algorithms
Neural networks
Medical problems
Genes
Glass

Keywords

  • Neural network ensemble
  • Backpropagation algorithm
  • Negative correlation learning
  • Constructive algorithms
  • Pruning algorithms

Cite this

@article{c787ef3ef9744a0f98ec0ecded84927b,
title = "A Dynamic Ensemble Learning Algorithm for Neural Networks",
abstract = "This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.",
keywords = "Neural network ensemble, Backpropagation algorithm, Negative correlation learning, Constructive algorithms, Pruning algorithms",
author = "N Siddique and Alam, {Kazi Md. Rokibul} and Hojjat Adeli",
year = "2019",
month = "7",
day = "28",
doi = "10.1007/s00521-019-04359-7",
language = "English",
journal = "Neural Computing and Applications",
issn = "0941-0643",

}

A Dynamic Ensemble Learning Algorithm for Neural Networks. / Siddique, N; Alam, Kazi Md. Rokibul ; Adeli, Hojjat.

In: Neural Computing and Applications, 28.07.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A Dynamic Ensemble Learning Algorithm for Neural Networks

AU - Siddique, N

AU - Alam, Kazi Md. Rokibul

AU - Adeli, Hojjat

PY - 2019/7/28

Y1 - 2019/7/28

N2 - This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.

AB - This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.

KW - Neural network ensemble

KW - Backpropagation algorithm

KW - Negative correlation learning

KW - Constructive algorithms

KW - Pruning algorithms

UR - http://www.scopus.com/inward/record.url?scp=85069898309&partnerID=8YFLogxK

U2 - 10.1007/s00521-019-04359-7

DO - 10.1007/s00521-019-04359-7

M3 - Article

JO - Neural Computing and Applications

T2 - Neural Computing and Applications

JF - Neural Computing and Applications

SN - 0941-0643

SN - 1433-3058

ER -