Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery

Pasquale Arpaia; Damien Coyle; Francesco Donnarumma; Antonio Esposito; Angela Natalizio; Marco Parvis; Marisa Pesola; Ersilia Vallefuoco

doi:10.1109/metroxraine54828.2022.9967501

Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery

Pasquale Arpaia
, Damien Coyle
, Francesco Donnarumma
, Antonio Esposito
, Angela Natalizio
, Marco Parvis
, Marisa Pesola
, Ersilia Vallefuoco

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Citations (Scopus)

Abstract

A multimodal sensory feedback was exploited in the present study to improve the detection of neurological phenomena associated with motor imagery. At this aim, visual and haptic feedback were simultaneously delivered to the user of a brain-computer interface. The motor imagery-based brain-computer interface was built by using a wearable and portable electroencephalograph with only eight dry electrodes, a haptic suit, and a purposely implemented virtual reality application. Preliminary experiments were carried out with six subjects participating in five sessions on different days. The subjects were randomly divided into “control group” and “neurofeedback group”. The former performed pure motor imagery without receiving any feedback, while the latter received multimodal feedback as a response to their imaginative act. Results of a cross validation showed that at most 61% of classification accuracy was achieved in performing the pure motor imagination. On the contrary, subjects of the “neurofeedback group” achieved up to 82% mean accuracy, with a peak of 91% in one of the sessions. However, no improvement in pure motor imagery was observed, either when practicing with pure motor imagery or with feedback.

Original language	English
Title of host publication	2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings
Publisher	IEEE
Pages	691-696
Number of pages	6
ISBN (Electronic)	978-1-6654-8574-6, 978-1-6654-8573-9
ISBN (Print)	978-1-6654-8575-3
DOIs	https://doi.org/10.1109/metroxraine54828.2022.9967501
Publication status	Published (in print/issue) - 5 Dec 2022
Event	IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) - Rome, Italy Duration: 26 Oct 2022 → 28 Oct 2022

Conference

Conference	IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)
Abbreviated title	MetroXRAINE
Country/Territory	Italy
City	Rome
Period	26/10/22 → 28/10/22

Bibliographical note

Funding Information:
This work was carried out as part of the "ICT for Health" project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Emanuele Cirillo, Stefania Di Rienzo, and Bianca Sorvillo for supporting the experiments.

Publisher Copyright:
© 2022 IEEE.

Funding

Funding Information: This work was carried out as part of the "ICT for Health" project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Emanuele Cirillo, Stefania Di Rienzo, and Bianca Sorvillo for supporting the experiments. Publisher Copyright: © 2022 IEEE.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 17 Partnerships for the Goals

Keywords

brain-computer interface
motor imagery
electroencephalography
extended reality
neurofeedback
wearability
dry electrodes
dry elecrtrodes

Access to Document

10.1109/metroxraine54828.2022.9967501

Cite this

Arpaia, P., Coyle, D., Donnarumma, F., Esposito, A., Natalizio, A., Parvis, M., Pesola, M., & Vallefuoco, E. (2022). Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery. In 2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings (pp. 691-696). IEEE. https://doi.org/10.1109/metroxraine54828.2022.9967501

@inproceedings{c849b3cf76024d0b8a07d5054c507850,

title = "Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery",

abstract = "A multimodal sensory feedback was exploited in the present study to improve the detection of neurological phenomena associated with motor imagery. At this aim, visual and haptic feedback were simultaneously delivered to the user of a brain-computer interface. The motor imagery-based brain-computer interface was built by using a wearable and portable electroencephalograph with only eight dry electrodes, a haptic suit, and a purposely implemented virtual reality application. Preliminary experiments were carried out with six subjects participating in five sessions on different days. The subjects were randomly divided into “control group” and “neurofeedback group”. The former performed pure motor imagery without receiving any feedback, while the latter received multimodal feedback as a response to their imaginative act. Results of a cross validation showed that at most 61\% of classification accuracy was achieved in performing the pure motor imagination. On the contrary, subjects of the “neurofeedback group” achieved up to 82\% mean accuracy, with a peak of 91\% in one of the sessions. However, no improvement in pure motor imagery was observed, either when practicing with pure motor imagery or with feedback.",

keywords = "brain-computer interface, motor imagery, electroencephalography, extended reality, neurofeedback, wearability, dry electrodes, dry elecrtrodes",

author = "Pasquale Arpaia and Damien Coyle and Francesco Donnarumma and Antonio Esposito and Angela Natalizio and Marco Parvis and Marisa Pesola and Ersilia Vallefuoco",

note = "Funding Information: This work was carried out as part of the {"}ICT for Health{"} project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative {\textquoteleft}Departments of Excellence{\textquoteright} (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Emanuele Cirillo, Stefania Di Rienzo, and Bianca Sorvillo for supporting the experiments. Publisher Copyright: {\textcopyright} 2022 IEEE.; IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE), MetroXRAINE ; Conference date: 26-10-2022 Through 28-10-2022",

year = "2022",

month = dec,

day = "5",

doi = "10.1109/metroxraine54828.2022.9967501",

language = "English",

isbn = "978-1-6654-8575-3",

pages = "691--696",

booktitle = "2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings",

publisher = "IEEE",

address = "United States",

}

Arpaia, P, Coyle, D, Donnarumma, F, Esposito, A, Natalizio, A, Parvis, M, Pesola, M & Vallefuoco, E 2022, Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery. in 2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings. IEEE, pp. 691-696, IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE), Rome, Italy, 26/10/22. https://doi.org/10.1109/metroxraine54828.2022.9967501

Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery. / Arpaia, Pasquale; Coyle, Damien; Donnarumma, Francesco et al.
2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings. IEEE, 2022. p. 691-696.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery

AU - Arpaia, Pasquale

AU - Coyle, Damien

AU - Donnarumma, Francesco

AU - Esposito, Antonio

AU - Natalizio, Angela

AU - Parvis, Marco

AU - Pesola, Marisa

AU - Vallefuoco, Ersilia

N1 - Funding Information: This work was carried out as part of the "ICT for Health" project, which was financially supported by the Italian Ministry of Education, University and Research (MIUR), under the initiative ‘Departments of Excellence’ (Italian Budget Law no. 232/2016), through an excellence grant awarded to the Department of Information Technology and Electrical Engineering of the University of Naples Federico II, Naples, Italy. The authors thank also thank Emanuele Cirillo, Stefania Di Rienzo, and Bianca Sorvillo for supporting the experiments. Publisher Copyright: © 2022 IEEE.

PY - 2022/12/5

Y1 - 2022/12/5

N2 - A multimodal sensory feedback was exploited in the present study to improve the detection of neurological phenomena associated with motor imagery. At this aim, visual and haptic feedback were simultaneously delivered to the user of a brain-computer interface. The motor imagery-based brain-computer interface was built by using a wearable and portable electroencephalograph with only eight dry electrodes, a haptic suit, and a purposely implemented virtual reality application. Preliminary experiments were carried out with six subjects participating in five sessions on different days. The subjects were randomly divided into “control group” and “neurofeedback group”. The former performed pure motor imagery without receiving any feedback, while the latter received multimodal feedback as a response to their imaginative act. Results of a cross validation showed that at most 61% of classification accuracy was achieved in performing the pure motor imagination. On the contrary, subjects of the “neurofeedback group” achieved up to 82% mean accuracy, with a peak of 91% in one of the sessions. However, no improvement in pure motor imagery was observed, either when practicing with pure motor imagery or with feedback.

AB - A multimodal sensory feedback was exploited in the present study to improve the detection of neurological phenomena associated with motor imagery. At this aim, visual and haptic feedback were simultaneously delivered to the user of a brain-computer interface. The motor imagery-based brain-computer interface was built by using a wearable and portable electroencephalograph with only eight dry electrodes, a haptic suit, and a purposely implemented virtual reality application. Preliminary experiments were carried out with six subjects participating in five sessions on different days. The subjects were randomly divided into “control group” and “neurofeedback group”. The former performed pure motor imagery without receiving any feedback, while the latter received multimodal feedback as a response to their imaginative act. Results of a cross validation showed that at most 61% of classification accuracy was achieved in performing the pure motor imagination. On the contrary, subjects of the “neurofeedback group” achieved up to 82% mean accuracy, with a peak of 91% in one of the sessions. However, no improvement in pure motor imagery was observed, either when practicing with pure motor imagery or with feedback.

KW - brain-computer interface

KW - motor imagery

KW - electroencephalography

KW - extended reality

KW - neurofeedback

KW - wearability

KW - dry electrodes

KW - dry elecrtrodes

UR - https://www.scopus.com/pages/publications/85144599557

U2 - 10.1109/metroxraine54828.2022.9967501

DO - 10.1109/metroxraine54828.2022.9967501

M3 - Conference contribution

SN - 978-1-6654-8575-3

SP - 691

EP - 696

BT - 2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings

PB - IEEE

T2 - IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)

Y2 - 26 October 2022 through 28 October 2022

ER -

Arpaia P, Coyle D, Donnarumma F, Esposito A, Natalizio A, Parvis M et al. Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery. In 2022 IEEE International Workshop on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering, MetroXRAINE 2022 - Proceedings. IEEE. 2022. p. 691-696 doi: 10.1109/metroxraine54828.2022.9967501

Multimodal Feedback in Assisting a Wearable Brain-Computer Interface Based on Motor Imagery

Abstract

Conference

Bibliographical note

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this