Multi-view Geometry Consistency Network for Facial Micro-Expression Recognition From Various Perspectives

Yawen Lu, Nikola Kasabov, Guyou Lu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

171 Downloads (Pure)


Micro-expression can reveal underlying genuine emotions, but those rapid and subtle changes are hard to be captured by humans. Most existing research focuses on frontal face micro-expression recognition, which largely prevents the developed methods from the real applications and ignores the underlying geometry information. In this paper, we propose a multiview geometry consistency framework to enable the same emotion to be recognized under different perspectives, which is difficult for existing systems. Based on the developed 3D face reconstruction network, the multi-view micro-expression recognition framework empowers the emotion recognition capability to learn from multiple perspectives of the 3D reconstructed faces based on view-consistency, and a spiking neural network is further applied to capture omitted tiny and detailed changes. With a sequence of images, we explore the subtle changes across frames through optical flow, which, as a clue, enhances the performance of our designated network for micro-expression recognition. Extensive experiments on benchmark micro-expression datasets CAS(ME) 2 and SMIC demonstrate the proposed method achieves promising results on novel-view micro-expression recognition where existing methods mainly fail.
Original languageEnglish
Title of host publication2021 International Joint Conference on Neural Networks (IJCNN)
PublisherIEEE Computational Intelligence Society
Number of pages8
ISBN (Electronic)978-1-6654-3900-8
ISBN (Print)978-1-6654-4597-9
Publication statusPublished (in print/issue) - 20 Sept 2021

Publication series

ISSN (Print)2161-4393
ISSN (Electronic)2161-4407


  • Micro-expression Recognition
  • 3D Face Reconstruction
  • Multiple View Geometry
  • Spiking Neural Networks


Dive into the research topics of 'Multi-view Geometry Consistency Network for Facial Micro-Expression Recognition From Various Perspectives'. Together they form a unique fingerprint.

Cite this