Prediction and Detection of Virtual Reality induced Cybersickness: A Spiking Neural Network Approach Using Spatiotemporal EEG Brain Data and Heart Rate Variability

Alexander Hui Xiang Yang, Nikola Kirilov Kasabov, Yusuf Ozgur Cakmak

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
47 Downloads (Pure)

Abstract

Virtual Reality (VR) allows users to interact with 3D immersive environments and has the potential to be a key technology across many domain applications, including access to a future metaverse. Yet, consumer adoption of VR technology is limited by cybersickness (CS)—a debilitating sensation accompanied by a cluster of symptoms, including nausea, oculomotor issues and dizziness. A leading problem is the lack of automated objective tools to predict or detect CS in individuals, which can then be used for resistance training, timely warning systems or clinical intervention. This paper explores the spatiotemporal brain dynamics and heart rate variability involved in cybersickness and uses this information to both predict and detect CS episodes. The present study applies deep learning of EEG in a spiking neural network (SNN) architecture to predict CS prior to using VR (85.9%, F7) and detect it (76.6%, FP1, Cz). ECG-derived sympathetic heart rate variability (HRV) parameters can be used for both prediction (74.2%) and detection (72.6%) but at a lower accuracy than EEG. Multimodal data fusion of EEG and sympathetic HRV does not change this accuracy compared to ECG alone. The study found that Cz (premotor and supplementary motor cortex) and O2 (primary visual cortex) are key hubs in functionally connected networks associated with both CS events and susceptibility to CS. F7 is also suggested as a key area involved in integrating information and implementing responses to incongruent environments that induce cybersickness. Consequently, Cz, O2 and F7 are presented here as promising targets for intervention.

Original languageEnglish
Article number15 (2023)
Pages (from-to)1-23
Number of pages23
JournalBrain Informatics
Volume10
Issue number15
Early online date12 Jul 2023
DOIs
Publication statusPublished online - 12 Jul 2023

Bibliographical note

Funding Information:
Authors thanks to New Zealand eScience Infrastructure (NeSI) team for providing the high capacity computing to extend our analyses. Thanks to Murray Cadzow for helping to port code into the NeSI infrastructure. Thanks to Sugam Bhudraja for discussions, providing initial code bases and help with Neucube.

Publisher Copyright:
© 2023, The Author(s).

Keywords

  • Cybersickness
  • Detection
  • Prediction
  • Biometrics
  • Physiological
  • Machine learning
  • AI
  • Neural networks
  • Virtual reality
  • Extended reality
  • Simulator
  • EEG
  • ECG
  • HRV
  • Spiking neural network
  • Brain
  • Dynamics
  • Spatiotemporal
  • NeuCube

Fingerprint

Dive into the research topics of 'Prediction and Detection of Virtual Reality induced Cybersickness: A Spiking Neural Network Approach Using Spatiotemporal EEG Brain Data and Heart Rate Variability'. Together they form a unique fingerprint.

Cite this