Classification of imagined spoken word-pairs using convolutional neural networks

Ciaran Cooney; Attila Korik; Folli Raffaella; Damien Coyle

doi:10.3217/978-3-85125-682-6-62

Classification of imagined spoken word-pairs using convolutional neural networks

Ciaran Cooney, Attila Korik, Folli Raffaella, Damien Coyle

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

550 Downloads (Pure)

Abstract

Imagined speech is gaining traction as a communicative paradigm for brain-computer-interfaces (BCI), as a growing body of research indicates the potential for decoding speech processes directly from the brain. The development of this type of direct-speech BCI has primarily considered feature extraction and machine learning approaches typical to BCI decoding. Here, we consider the potential of deep learning as a possible alternative to traditional BCI methodologies in relation to imagined speech EEG decoding. Two different convolutional neural networks (CNN) were trained on multiple imagined speech word-pairs, and their performance compared to a baseline linear discriminant analysis (LDA) classifier trained using filterbank common spatial patterns (FBCSP) features. Classifiers were trained using nested cross-validation to enable hyper-parameter optimization. Results obtained showed that the CNNs outperformed the FBCSP with average accuracies of 62.37% and 60.88% vs. 57.80% (p<0.005).

Original language	English
Title of host publication	Proceedings of the 8th Graz Brain Computer Interface Conference 2019
Subtitle of host publication	Bridging Science and Application
Editors	Gernot R Muller-Putz, Jonas C Ditz, Selina C Wriessnegger
Publisher	Verlag der Technischen Universitat Graz
Pages	338-343
Volume	2019
ISBN (Print)	978-3-85125-682-6
DOIs	https://doi.org/10.3217/978-3-85125-682-6-62
Publication status	Published (in print/issue) - 20 Sept 2019
Event	The 8th Graz BCI Conference, 2019 - Institute of Neural Engineering, Graz University of Technology, Graz, Austria Duration: 16 Sept 2019 → 20 Sept 2023 https://www.tugraz.at/institute/ine/graz-bci-conferences/8th-graz-bci-conference-2019/

Publication series

Name	Proceedings of the 8th Graz Brain-Computer Interface Conference 2019
Publisher	Graz University of Technology
ISSN (Print)	2311-0422

Conference

Conference	The 8th Graz BCI Conference, 2019
Country/Territory	Austria
City	Graz
Period	16/09/19 → 20/09/23
Internet address	https://www.tugraz.at/institute/ine/graz-bci-conferences/8th-graz-bci-conference-2019/

Keywords

Electroencephalogram (EEG)
Imagined Speech
Convolutional Neural Network
Brain-Computer Interface

Access to Document

10.3217/978-3-85125-682-6-62Licence: CC BY-ND

graz_resubmissionAuthor Accepted Manuscript, 382 KBLicence: CC BY-ND

Cite this

Cooney, C., Korik, A., Raffaella, F., & Coyle, D. (2019). Classification of imagined spoken word-pairs using convolutional neural networks. In G. R. Muller-Putz, J. C. Ditz, & S. C. Wriessnegger (Eds.), Proceedings of the 8th Graz Brain Computer Interface Conference 2019: Bridging Science and Application (Vol. 2019, pp. 338-343). (Proceedings of the 8th Graz Brain-Computer Interface Conference 2019). Verlag der Technischen Universitat Graz. https://doi.org/10.3217/978-3-85125-682-6-62

Cooney, Ciaran ; Korik, Attila ; Raffaella, Folli et al. / Classification of imagined spoken word-pairs using convolutional neural networks. Proceedings of the 8th Graz Brain Computer Interface Conference 2019: Bridging Science and Application. editor / Gernot R Muller-Putz ; Jonas C Ditz ; Selina C Wriessnegger. Vol. 2019 Verlag der Technischen Universitat Graz, 2019. pp. 338-343 (Proceedings of the 8th Graz Brain-Computer Interface Conference 2019).

@inproceedings{7586ba4f3f044f7b859f1e6696999320,

title = "Classification of imagined spoken word-pairs using convolutional neural networks",

abstract = " Imagined speech is gaining traction as a communicative paradigm for brain-computer-interfaces (BCI), as a growing body of research indicates the potential for decoding speech processes directly from the brain. The development of this type of direct-speech BCI has primarily considered feature extraction and machine learning approaches typical to BCI decoding. Here, we consider the potential of deep learning as a possible alternative to traditional BCI methodologies in relation to imagined speech EEG decoding. Two different convolutional neural networks (CNN) were trained on multiple imagined speech word-pairs, and their performance compared to a baseline linear discriminant analysis (LDA) classifier trained using filterbank common spatial patterns (FBCSP) features. Classifiers were trained using nested cross-validation to enable hyper-parameter optimization. Results obtained showed that the CNNs outperformed the FBCSP with average accuracies of 62.37% and 60.88% vs. 57.80% (p<0.005).",

keywords = "Electroencephalogram (EEG), Imagined Speech, Convolutional Neural Network, Brain-Computer Interface",

author = "Ciaran Cooney and Attila Korik and Folli Raffaella and Damien Coyle",

year = "2019",

month = sep,

day = "20",

doi = "10.3217/978-3-85125-682-6-62",

language = "English",

isbn = "978-3-85125-682-6 ",

volume = "2019",

series = "Proceedings of the 8th Graz Brain-Computer Interface Conference 2019",

publisher = "Verlag der Technischen Universitat Graz",

pages = "338--343",

editor = "Muller-Putz, {Gernot R} and Ditz, {Jonas C} and Wriessnegger, {Selina C}",

booktitle = "Proceedings of the 8th Graz Brain Computer Interface Conference 2019",

note = "The 8th Graz BCI Conference, 2019 ; Conference date: 16-09-2019 Through 20-09-2023",

url = "https://www.tugraz.at/institute/ine/graz-bci-conferences/8th-graz-bci-conference-2019/",

}

Cooney, C, Korik, A , Raffaella, F & Coyle, D 2019, Classification of imagined spoken word-pairs using convolutional neural networks. in GR Muller-Putz, JC Ditz & SC Wriessnegger (eds), Proceedings of the 8th Graz Brain Computer Interface Conference 2019: Bridging Science and Application. vol. 2019, Proceedings of the 8th Graz Brain-Computer Interface Conference 2019, Verlag der Technischen Universitat Graz, pp. 338-343, The 8th Graz BCI Conference, 2019, Graz, Austria, 16/09/19. https://doi.org/10.3217/978-3-85125-682-6-62

Classification of imagined spoken word-pairs using convolutional neural networks. / Cooney, Ciaran; Korik, Attila ; Raffaella, Folli et al.
Proceedings of the 8th Graz Brain Computer Interface Conference 2019: Bridging Science and Application. ed. / Gernot R Muller-Putz; Jonas C Ditz; Selina C Wriessnegger. Vol. 2019 Verlag der Technischen Universitat Graz, 2019. p. 338-343 (Proceedings of the 8th Graz Brain-Computer Interface Conference 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Classification of imagined spoken word-pairs using convolutional neural networks

AU - Cooney, Ciaran

AU - Korik, Attila

AU - Raffaella, Folli

AU - Coyle, Damien

PY - 2019/9/20

Y1 - 2019/9/20

N2 - Imagined speech is gaining traction as a communicative paradigm for brain-computer-interfaces (BCI), as a growing body of research indicates the potential for decoding speech processes directly from the brain. The development of this type of direct-speech BCI has primarily considered feature extraction and machine learning approaches typical to BCI decoding. Here, we consider the potential of deep learning as a possible alternative to traditional BCI methodologies in relation to imagined speech EEG decoding. Two different convolutional neural networks (CNN) were trained on multiple imagined speech word-pairs, and their performance compared to a baseline linear discriminant analysis (LDA) classifier trained using filterbank common spatial patterns (FBCSP) features. Classifiers were trained using nested cross-validation to enable hyper-parameter optimization. Results obtained showed that the CNNs outperformed the FBCSP with average accuracies of 62.37% and 60.88% vs. 57.80% (p<0.005).

AB - Imagined speech is gaining traction as a communicative paradigm for brain-computer-interfaces (BCI), as a growing body of research indicates the potential for decoding speech processes directly from the brain. The development of this type of direct-speech BCI has primarily considered feature extraction and machine learning approaches typical to BCI decoding. Here, we consider the potential of deep learning as a possible alternative to traditional BCI methodologies in relation to imagined speech EEG decoding. Two different convolutional neural networks (CNN) were trained on multiple imagined speech word-pairs, and their performance compared to a baseline linear discriminant analysis (LDA) classifier trained using filterbank common spatial patterns (FBCSP) features. Classifiers were trained using nested cross-validation to enable hyper-parameter optimization. Results obtained showed that the CNNs outperformed the FBCSP with average accuracies of 62.37% and 60.88% vs. 57.80% (p<0.005).

KW - Electroencephalogram (EEG)

KW - Imagined Speech

KW - Convolutional Neural Network

KW - Brain-Computer Interface

UR - http://diglib.tugraz.at/proceedings-of-the-8th-graz-brain-computer-interface-conference-2019-bridging-science-and-application-2019

U2 - 10.3217/978-3-85125-682-6-62

DO - 10.3217/978-3-85125-682-6-62

M3 - Conference contribution

SN - 978-3-85125-682-6

VL - 2019

T3 - Proceedings of the 8th Graz Brain-Computer Interface Conference 2019

SP - 338

EP - 343

BT - Proceedings of the 8th Graz Brain Computer Interface Conference 2019

A2 - Muller-Putz, Gernot R

A2 - Ditz, Jonas C

A2 - Wriessnegger, Selina C

PB - Verlag der Technischen Universitat Graz

T2 - The 8th Graz BCI Conference, 2019

Y2 - 16 September 2019 through 20 September 2023

ER -

Cooney C, Korik A , Raffaella F , Coyle D. Classification of imagined spoken word-pairs using convolutional neural networks. In Muller-Putz GR, Ditz JC, Wriessnegger SC, editors, Proceedings of the 8th Graz Brain Computer Interface Conference 2019: Bridging Science and Application. Vol. 2019. Verlag der Technischen Universitat Graz. 2019. p. 338-343. (Proceedings of the 8th Graz Brain-Computer Interface Conference 2019). doi: 10.3217/978-3-85125-682-6-62

Classification of imagined spoken word-pairs using convolutional neural networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this