Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

S. Dora; K. Subramanian; S. Suresh; N. Sundararajan

doi:10.1016/j.neucom.2015.07.086

Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

S. Dora
, K. Subramanian
, S. Suresh
, N. Sundararajan

Nanyang Technological University

Research output: Contribution to journal › Article › peer-review

83 Citations (Scopus)

Abstract

This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.

Original language	English
Pages (from-to)	1216-1229
Number of pages	14
Journal	Neurocomputing
Volume	171
Early online date	10 Aug 2015
DOIs	https://doi.org/10.1016/j.neucom.2015.07.086
Publication status	Published (in print/issue) - 1 Jan 2016

Funding

The authors would like to acknowledge the reviewers for their invaluable comments, which has helped to improve the quality of this paper. Shirin Dora received his B.Tech. degree in electronics and instrumentation engineering from Mahrishi Dayanand University, Rohtak, India in 2007. Currently, he is working towards his Ph.D. with the School of Computer Engineering from Nanyang Technological University, Singapore. His research interests include neural networks, machine learning and Neuromorphic systems. Kartick Subramanian received his M.Sc. and Ph.D. from Nanyang Technological University, Singapore. He is currently working as a Post-Doctoral Researcher with School of Computer Engineering, Nanyang Technological University. His research interest includes neural networks, fuzzy logic-based system, machine learning and video analytics. Sundaram Suresh received the B.E degree in electrical and electronics engineering from Bharathiyar University in 1999, and the M.E. and Ph.D. degrees in aerospace engineering from the Indian Institute of Science, Bangalore, India, in 2001 and 2005, respectively. He was a Post-Doctoral Researcher in the School of Electrical Engineering, Nanyang Technological University from 2005 to 2007. From 2007 to 2008, he was with the National Institute for Research in Computer Science and Control-Sophia Antipolis, Nice, France as Research Fellow of the European Research Consortium for Informatics and Mathematics. He was with Korea University, Seoul, Korea, for a short period as a visiting faculty in Industrial Engineering. From January 2009 to December 2009, he was with the Department of Electrical Engineering, Indian Institute of Technology, Delhi, India, as an Assistant Professor. Since 2010, he has been an Assistant Professor with the School of Computer Engineering, Nanyang Technological University. His research interest includes flight control, unmanned aerial vehicle design, machine learning, and optimization and computer vision. Narasimhan Sundararajan received the B.E. degree in electrical engineering (with first-class Hons.) from the University of Madras, Chennai, India, in 1966, M.Tech. degree from the Indian institute of Technology, Madras, in 1968 and the Ph.D. degree in electrical engineering from the University of Illinois at Urbana-Champaign, Urbana, IL, USA, in 1971. He has worked in various capacities with the Indian Space Research Organization, Bengaluru, India, since 1971. Since 1991, he has been a Professor with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. He was a National Research Council Research Associate with the National Aeronautics and Space Administration (NASA) Ames Research Center, Ames, CA, USA, in 1974 and a Senior Research Associate with NASA Langley, Hampton, VA, USA between 1981 and 1986. He has published more than 250 papers and written six books in the field of computational intelligence and neural networks. His current research interests include aerospace control and neural networks. Dr. Sundararajan is an Associate Fellow of the American Institute of Aeronautics and Astronautics and a Fellow of the Institution of Engineers, Singapore.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Pattern classification
Self-regulation
Spiking neural network

Access to Document

10.1016/j.neucom.2015.07.086

Cite this

@article{614a3983bf0349f487a0600666192466,

title = "Development of a Self-Regulating Evolving Spiking Neural Network for classification problem",

abstract = "This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.",

keywords = "Pattern classification, Self-regulation, Spiking neural network",

author = "S. Dora and K. Subramanian and S. Suresh and N. Sundararajan",

year = "2016",

month = jan,

day = "1",

doi = "10.1016/j.neucom.2015.07.086",

language = "English",

volume = "171",

pages = "1216--1229",

journal = "Neurocomputing",

issn = "0925-2312",

publisher = "Elsevier BV",

}

TY - JOUR

T1 - Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

AU - Dora, S.

AU - Subramanian, K.

AU - Suresh, S.

AU - Sundararajan, N.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.

AB - This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.

KW - Pattern classification

KW - Self-regulation

KW - Spiking neural network

UR - https://pure.ulster.ac.uk/en/publications/development-of-a-self-regulating-evolving-spiking-neural-network-

UR - http://www.scopus.com/inward/record.url?scp=84944474611&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2015.07.086

DO - 10.1016/j.neucom.2015.07.086

M3 - Article

AN - SCOPUS:84944474611

SN - 0925-2312

VL - 171

SP - 1216

EP - 1229

JO - Neurocomputing

JF - Neurocomputing

ER -

Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

Abstract

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this