Integrating Spatial and Temporal Information for Violent Activity Detection from Video Using Deep Spiking Neural Networks

Xiang Wang, Jie Yang, Nikola Kasabov

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
18 Downloads (Pure)

Abstract

Increasing violence in workplaces such as hospitals seriously challenges public safety. However, it is time- and labor-consuming to visually monitor masses of video data in real time. Therefore, automatic and timely violent activity detection from videos is vital, especially for small monitoring systems. This paper proposes a two-stream deep learning architecture for video violent activity detection named SpikeConvFlowNet. First, RGB frames and their optical flow data are used as inputs for each stream to extract the spatiotemporal features of videos. After that, the spatiotemporal features from the two streams are concatenated and fed to the classifier for the final decision. Each stream utilizes a supervised neural network consisting of multiple convolutional spiking and pooling layers. Convolutional layers are used to extract high-quality spatial features within frames, and spiking neurons can efficiently extract temporal features across frames by remembering historical information. The spiking neuron-based optical flow can strengthen the capability of extracting critical motion information. This method combines their advantages to enhance the performance and
efficiency for recognizing violent actions. The experimental results on public datasets demonstrate that, compared with the latest methods, this approach greatly reduces parameters and achieves higher inference efficiency with limited accuracy loss. It is a potential solution for applications in embedded devices that provide low computing power but require fast processing speeds.
Original languageEnglish
Article number4532
Pages (from-to)1-17
Number of pages17
JournalSensors
Volume23
Issue number9
DOIs
Publication statusPublished (in print/issue) - 6 May 2023

Bibliographical note

Publisher Copyright:
© 2023 by the authors.

Keywords

  • violence detection
  • spiking neural networks
  • deep learning
  • optical flow
  • spatial and temporal analysis
  • spiking neural network

Fingerprint

Dive into the research topics of 'Integrating Spatial and Temporal Information for Violent Activity Detection from Video Using Deep Spiking Neural Networks'. Together they form a unique fingerprint.

Cite this