Improving SpikeProp’s Training Efficiency in Spiking Neural Networks for Large Language Models Through Innovative Weight Initialization

Spiking neural networks (SNNs) use individual temporal spikes for computation and communication, simulating the actions of biological neurons. SNN had long been disregarded since it was thought to be intricate and difficult to analyze. We investigate the improvement of SpikeProp, a supervised learning model tailored for SNNs, in this work. Three distinct models are being proposed and investigated, including the proposed model 1, the proposed model 2, and the proposed model 3, each providing unique improvements to the SpikeProp algorithm. To accelerate convergence and adaptive learning rates, particle swarm optimization (PSO) and momentum factors are integrated into the proposed model 1. In proposed model 2, a rate dependency is introduced based on angle-driven learning. By incorporating PSO and learning rates, model 3 combines the strengths of both models 1 and 2. We believe, SNNs can be trained and classified more efficiently and accurately using these models. Furthermore, we examine how large language models (LLMs) might inform the design and interpretability of neural architectures and learning methodologies while also enhancing SNN training. Through the use of LLMs, we seek to enhance model transparency and encourage more Responsible AI (RAI) principles. A thorough evaluation and comparison of proposed models with traditional methods confirms that these models consistently outperform traditional methods for various real datasets. Consequently, they have a high potential for practical applications in neural network training in real-world settings and LLM-informed development, contributing to the advancement of AI systems.