Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces

pasquale  Arpaia; Damien Coyle; Francesca Donnarumma; University Naple; Angela Natalizio; Marco Parvis

doi:10.1007/978-3-031-15553-6_28

Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces

pasquale Arpaia
, Damien Coyle
, Francesca Donnarumma
, University Naple
, Angela Natalizio
, Marco Parvis

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Citations (Scopus)

262 Downloads (Pure)

Abstract

A sensory feedback was employed for the present work to remap brain signals into sensory information. In particular, sensorimotor rhythms associated with motor imagery were measured as a mean to interact with an extended reality (XR) environment. The aim for such a neurofeedback was to let the user become aware of his/her ability to imagine a movement. A brain-computer interface based on motor imagery was thus implemented by using a consumer-grade electroencephalograph and by taking into account wearable and portable feedback actuators. Visual and vibrotactile sensory feedback modalities were used simultaneously to provide an engaging multimodal feedback in XR. Both a non-immersive and an immersive version of the system were considered and compared. Preliminary validation was carried out with four healthy subjects participating in a total of four sessions on different days. Experiments were conducted according to a wide-spread synchronous paradigm in which an application provides the timing for the motor imagery tasks. Performance was compared in terms of classification accuracy. Overall, subjects preferred the immersive neurofeedback because it allowed higher concentration during experiments, but there was not enough evidence to prove its actual effectiveness and mean classification accuracy resulted about 65%. Meanwhile, classification accuracy resulted higher with the non-immersive neurofeedback, notably it reached about 75%. Future experiments could extend this comparison to more subjects and more sessions, due to the relevance of possible applications in rehabilitation. Moreover, the immersive XR implementation could be improved to provide a greater sense of embodiment.

Original language	English
Title of host publication	International Conference on Extended Reality
Editors	Lucio Tommaso De Paolis, Pasquale Arpaia, Marco Sacco
Publisher	Springer Nature Switzerland AG
Pages	407-419
Number of pages	12
Volume	13446
ISBN (Electronic)	978-3-031-15553-6
ISBN (Print)	978-3-031-15552-9
DOIs	https://doi.org/10.1007/978-3-031-15553-6_28
Publication status	Published (in print/issue) - 28 Aug 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13446 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Bibliographical note

Funding Information:
Acknowledgement. This work was carried out as part of the “ICT for Health” project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Giovanni D’Errico and Stefania Di Rienzo for supporting system design and data analyses.

Publisher Copyright:
© 2022, Springer Nature Switzerland AG.

Funding

Funding Information: Acknowledgement. This work was carried out as part of the “ICT for Health” project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Giovanni D’Errico and Stefania Di Rienzo for supporting system design and data analyses. Publisher Copyright: © 2022, Springer Nature Switzerland AG.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Brain computer interface
Extended Reality
Motor Imagery
Electroencephaography
Haptics
Neurofeedback
Brain-computer interface
Extended reality
Electroencephalography
Motor imagery

Access to Document

10.1007/978-3-031-15553-6_28

Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces published version DCFinal published version, 639 KB

Cite this

Arpaia, P., Coyle, D., Donnarumma, F., Naple, U., Natalizio, A., & Parvis, M. (2022). Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces. In L. T. De Paolis, P. Arpaia, & M. Sacco (Eds.), International Conference on Extended Reality (Vol. 13446, pp. 407-419). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13446 LNCS). Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-15553-6_28

Arpaia, pasquale ; Coyle, Damien ; Donnarumma, Francesca et al. / Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces. International Conference on Extended Reality. editor / Lucio Tommaso De Paolis ; Pasquale Arpaia ; Marco Sacco. Vol. 13446 Springer Nature Switzerland AG, 2022. pp. 407-419 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{deb4713a780d442084fab10ac5e544b4,

title = "Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces",

abstract = "A sensory feedback was employed for the present work to remap brain signals into sensory information. In particular, sensorimotor rhythms associated with motor imagery were measured as a mean to interact with an extended reality (XR) environment. The aim for such a neurofeedback was to let the user become aware of his/her ability to imagine a movement. A brain-computer interface based on motor imagery was thus implemented by using a consumer-grade electroencephalograph and by taking into account wearable and portable feedback actuators. Visual and vibrotactile sensory feedback modalities were used simultaneously to provide an engaging multimodal feedback in XR. Both a non-immersive and an immersive version of the system were considered and compared. Preliminary validation was carried out with four healthy subjects participating in a total of four sessions on different days. Experiments were conducted according to a wide-spread synchronous paradigm in which an application provides the timing for the motor imagery tasks. Performance was compared in terms of classification accuracy. Overall, subjects preferred the immersive neurofeedback because it allowed higher concentration during experiments, but there was not enough evidence to prove its actual effectiveness and mean classification accuracy resulted about 65\%. Meanwhile, classification accuracy resulted higher with the non-immersive neurofeedback, notably it reached about 75\%. Future experiments could extend this comparison to more subjects and more sessions, due to the relevance of possible applications in rehabilitation. Moreover, the immersive XR implementation could be improved to provide a greater sense of embodiment.",

keywords = "Brain computer interface, Extended Reality, Motor Imagery, Electroencephaography, Haptics, Neurofeedback, Brain-computer interface, Extended reality, Electroencephalography, Motor imagery",

author = "pasquale Arpaia and Damien Coyle and Francesca Donnarumma and University Naple and Angela Natalizio and Marco Parvis",

note = "Funding Information: Acknowledgement. This work was carried out as part of the “ICT for Health” project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative {\textquoteleft}Departments of Excellence{\textquoteright} (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Giovanni D{\textquoteright}Errico and Stefania Di Rienzo for supporting system design and data analyses. Publisher Copyright: {\textcopyright} 2022, Springer Nature Switzerland AG.",

year = "2022",

month = aug,

day = "28",

doi = "10.1007/978-3-031-15553-6\_28",

language = "English",

isbn = "978-3-031-15552-9",

volume = "13446",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Nature Switzerland AG",

pages = "407--419",

editor = "\{De Paolis\}, \{Lucio Tommaso\} and Pasquale Arpaia and Marco Sacco",

booktitle = "International Conference on Extended Reality",

address = "Switzerland",

}

Arpaia, P, Coyle, D, Donnarumma, F, Naple, U, Natalizio, A & Parvis, M 2022, Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces. in LT De Paolis, P Arpaia & M Sacco (eds), International Conference on Extended Reality. vol. 13446, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13446 LNCS, Springer Nature Switzerland AG, pp. 407-419. https://doi.org/10.1007/978-3-031-15553-6_28

Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces. / Arpaia, pasquale ; Coyle, Damien; Donnarumma, Francesca et al.
International Conference on Extended Reality. ed. / Lucio Tommaso De Paolis; Pasquale Arpaia; Marco Sacco. Vol. 13446 Springer Nature Switzerland AG, 2022. p. 407-419 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13446 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces

AU - Arpaia, pasquale

AU - Coyle, Damien

AU - Donnarumma, Francesca

AU - Naple, University

AU - Natalizio, Angela

AU - Parvis, Marco

N1 - Funding Information: Acknowledgement. This work was carried out as part of the “ICT for Health” project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Giovanni D’Errico and Stefania Di Rienzo for supporting system design and data analyses. Publisher Copyright: © 2022, Springer Nature Switzerland AG.

PY - 2022/8/28

Y1 - 2022/8/28

N2 - A sensory feedback was employed for the present work to remap brain signals into sensory information. In particular, sensorimotor rhythms associated with motor imagery were measured as a mean to interact with an extended reality (XR) environment. The aim for such a neurofeedback was to let the user become aware of his/her ability to imagine a movement. A brain-computer interface based on motor imagery was thus implemented by using a consumer-grade electroencephalograph and by taking into account wearable and portable feedback actuators. Visual and vibrotactile sensory feedback modalities were used simultaneously to provide an engaging multimodal feedback in XR. Both a non-immersive and an immersive version of the system were considered and compared. Preliminary validation was carried out with four healthy subjects participating in a total of four sessions on different days. Experiments were conducted according to a wide-spread synchronous paradigm in which an application provides the timing for the motor imagery tasks. Performance was compared in terms of classification accuracy. Overall, subjects preferred the immersive neurofeedback because it allowed higher concentration during experiments, but there was not enough evidence to prove its actual effectiveness and mean classification accuracy resulted about 65%. Meanwhile, classification accuracy resulted higher with the non-immersive neurofeedback, notably it reached about 75%. Future experiments could extend this comparison to more subjects and more sessions, due to the relevance of possible applications in rehabilitation. Moreover, the immersive XR implementation could be improved to provide a greater sense of embodiment.

AB - A sensory feedback was employed for the present work to remap brain signals into sensory information. In particular, sensorimotor rhythms associated with motor imagery were measured as a mean to interact with an extended reality (XR) environment. The aim for such a neurofeedback was to let the user become aware of his/her ability to imagine a movement. A brain-computer interface based on motor imagery was thus implemented by using a consumer-grade electroencephalograph and by taking into account wearable and portable feedback actuators. Visual and vibrotactile sensory feedback modalities were used simultaneously to provide an engaging multimodal feedback in XR. Both a non-immersive and an immersive version of the system were considered and compared. Preliminary validation was carried out with four healthy subjects participating in a total of four sessions on different days. Experiments were conducted according to a wide-spread synchronous paradigm in which an application provides the timing for the motor imagery tasks. Performance was compared in terms of classification accuracy. Overall, subjects preferred the immersive neurofeedback because it allowed higher concentration during experiments, but there was not enough evidence to prove its actual effectiveness and mean classification accuracy resulted about 65%. Meanwhile, classification accuracy resulted higher with the non-immersive neurofeedback, notably it reached about 75%. Future experiments could extend this comparison to more subjects and more sessions, due to the relevance of possible applications in rehabilitation. Moreover, the immersive XR implementation could be improved to provide a greater sense of embodiment.

KW - Brain computer interface

KW - Extended Reality

KW - Motor Imagery

KW - Electroencephaography

KW - Haptics

KW - Neurofeedback

KW - Brain-computer interface

KW - Extended reality

KW - Electroencephalography

KW - Motor imagery

UR - https://www.scopus.com/pages/publications/85138258070

U2 - 10.1007/978-3-031-15553-6_28

DO - 10.1007/978-3-031-15553-6_28

M3 - Conference contribution

SN - 978-3-031-15552-9

VL - 13446

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 407

EP - 419

BT - International Conference on Extended Reality

A2 - De Paolis, Lucio Tommaso

A2 - Arpaia, Pasquale

A2 - Sacco, Marco

PB - Springer Nature Switzerland AG

ER -

Arpaia P, Coyle D, Donnarumma F, Naple U, Natalizio A, Parvis M. Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces. In De Paolis LT, Arpaia P, Sacco M, editors, International Conference on Extended Reality. Vol. 13446. Springer Nature Switzerland AG. 2022. p. 407-419. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-15553-6_28

Non-immersive Versus Immersive Extended Reality for Motor Imagery Neurofeedback Within a Brain-Computer Interfaces

Abstract

Publication series

Bibliographical note

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this