Sensitivity-Based Adaptive Learning Rules for Binary Feedforward Neural Networks

Shuiming Zhong, Xiaoqin Zeng, Shengli Wu, Lixin Han

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.
LanguageEnglish
Pages480-491
JournalIEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Volume23
Issue number3
DOIs
Publication statusPublished - Mar 2012

Fingerprint

Feedforward neural networks
Adaptive algorithms
Learning algorithms
Backpropagation algorithms
Neurons

Cite this

Zhong, Shuiming ; Zeng, Xiaoqin ; Wu, Shengli ; Han, Lixin. / Sensitivity-Based Adaptive Learning Rules for Binary Feedforward Neural Networks. 2012 ; Vol. 23, No. 3. pp. 480-491.
@article{a1f17498c67b433bb314ae6be3fd141f,
title = "Sensitivity-Based Adaptive Learning Rules for Binary Feedforward Neural Networks",
abstract = "This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.",
author = "Shuiming Zhong and Xiaoqin Zeng and Shengli Wu and Lixin Han",
year = "2012",
month = "3",
doi = "10.1109/TNNLS.2011.2177860",
language = "English",
volume = "23",
pages = "480--491",
number = "3",

}

Sensitivity-Based Adaptive Learning Rules for Binary Feedforward Neural Networks. / Zhong, Shuiming; Zeng, Xiaoqin; Wu, Shengli; Han, Lixin.

Vol. 23, No. 3, 03.2012, p. 480-491.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Sensitivity-Based Adaptive Learning Rules for Binary Feedforward Neural Networks

AU - Zhong, Shuiming

AU - Zeng, Xiaoqin

AU - Wu, Shengli

AU - Han, Lixin

PY - 2012/3

Y1 - 2012/3

N2 - This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.

AB - This paper proposes a set of adaptive learning rules for binary feedforward neural networks (BFNNs) by means of the sensitivity measure that is established to investigate the effect of a BFNN's weight variation on its output. The rules are based on three basic adaptive learning principles: the benefit principle, the minimal disturbance principle, and the burden-sharing principle. In order to follow the benefit principle and the minimal disturbance principle, a neuron selection rule and a weight adaptation rule are developed. Besides, a learning control rule is developed to follow the burden-sharing principle. The advantage of the rules is that they can effectively guide the BFNN's learning to conduct constructive adaptations and avoid destructive ones. With these rules, a sensitivity-based adaptive learning (SBALR) algorithm for BFNNs is presented. Experimental results on a number of benchmark data demonstrate that the SBALR algorithm has better learning performance than the Madaline rule II and backpropagation algorithms.

U2 - 10.1109/TNNLS.2011.2177860

DO - 10.1109/TNNLS.2011.2177860

M3 - Article

VL - 23

SP - 480

EP - 491

IS - 3

ER -