Emotion recognition with convolutional neural network and EEG-based EFDMs

Fei Wang, Shichao Wu, Weiwei Zhang, Zongfeng Xu, Yahui Zhang, Chengdong Wu, Sonya Coleman

Research output: Contribution to journalArticlepeer-review

64 Citations (Scopus)
651 Downloads (Pure)


Electroencephalogram (EEG), as a direct response to brain activity, can be used to detect mental states and physical conditions. Among various EEG-based emotion recognition studies, due to the non-linear, non-stationary and the individual difference of EEG signals, traditional recognition methods still have the disadvantages of complicated feature extraction and low recognition rates. Thus, this paper first proposes a novel concept of electrode-frequency distribution maps (EFDMs) with short-time Fourier transform (STFT). Residual block based deep convolutional neural network (CNN) is proposed for automatic feature extraction and emotion classification with EFDMs. Aim at the shortcomings of the small amount of EEG samples and the challenge of differences in individual emotions, which makes it difficult to construct a universal model, this paper proposes a cross-datasets emotion recognition method of deep model transfer learning. Experiments carried out on two publicly available datasets. The proposed method achieved an average classification score of 90.59% based on a short length of EEG data on SEED, which is 4.51% higher than the baseline method. Then, the pre-trained model was applied to DEAP through deep model transfer learning with a few samples, resulted an average accuracy of 82.84%. Finally, this paper adopts the gradient weighted class activation mapping (Grad-CAM) to get a glimpse of what features the CNN has learned during training from EFDMs and concludes that the high frequency bands are more favorable for emotion recognition.
Original languageEnglish
Article number107506
Number of pages11
Early online date1 Jun 2020
Publication statusPublished (in print/issue) - 30 Sept 2020


  • Convolutional neural network
  • Electrode-frequency distribution maps
  • Electroencephalogram
  • Emotion recognition
  • Gradient-weighted class activation mapping


Dive into the research topics of 'Emotion recognition with convolutional neural network and EEG-based EFDMs'. Together they form a unique fingerprint.

Cite this