Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs

Nikin Matharaarachchi; Muhammad Fermi Pasha; Sonya Coleman; Dermot Kerr

doi:10.1109/ICMLA58977.2023.00055

Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs

Nikin Matharaarachchi
, Muhammad Fermi Pasha
, Sonya Coleman
, Dermot Kerr

Monash University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

70 Downloads (Pure)

Abstract

Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional Neural
Networks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers results
comparable to the state-of-the-art.

Original language	English
Title of host publication	2023 International Conference on Machine Learning and Applications (ICMLA)
Editors	M. Arif Wani, Mihai Boicu, Moamar Sayed-Mouchaweh, Pedro Henriques Abreu, Joao Gama
Publisher	IEEE
Pages	1-8
Number of pages	8
ISBN (Electronic)	979-8-3503-4534-6
ISBN (Print)	979-8-3503-1891-3
DOIs	https://doi.org/10.1109/ICMLA58977.2023.00055
Publication status	Published online - 19 Mar 2024
Event	22nd International Conference on Machine Learning and Applications - Hyatt Regency Jacksonville Riverfront, Florida, United States Duration: 15 Dec 2023 → 17 Dec 2023 https://www.icmla-conference.org/icmla23/

Publication series

Name	Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023

Conference

Conference	22nd International Conference on Machine Learning and Applications
Abbreviated title	ICMLA'23
Country/Territory	United States
City	Florida
Period	15/12/23 → 17/12/23
Internet address	https://www.icmla-conference.org/icmla23/

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Funding

This work was supported by the Malaysian Ministry of Higher Education via the Fundamental Research Grant Scheme (FRGS) project no.: FRGS/1/2020/ICT02/MUSM/03/4.

Funder number
FRGS/1/2020/ICT02/MUSM/03/4

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

micro-expressions
emotion
GCN
Deep learning
graph networks
deep learning

Access to Document

10.1109/ICMLA58977.2023.00055

ICMLA_Time_Efficient_Micro_Expression_Recognition_using_Weighted_Spatio_Temporal_Landmark_Graphs (11) (1)Author Accepted Manuscript, 1.11 MB

Cite this

Matharaarachchi, N., Pasha, M. F., Coleman, S., & Kerr, D. (2024). Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs. In M. Arif Wani, M. Boicu, M. Sayed-Mouchaweh, P. H. Abreu, & J. Gama (Eds.), 2023 International Conference on Machine Learning and Applications (ICMLA) (pp. 1-8). (Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023). IEEE. Advance online publication. https://doi.org/10.1109/ICMLA58977.2023.00055

Matharaarachchi, Nikin ; Pasha, Muhammad Fermi ; Coleman, Sonya et al. / Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs. 2023 International Conference on Machine Learning and Applications (ICMLA). editor / M. Arif Wani ; Mihai Boicu ; Moamar Sayed-Mouchaweh ; Pedro Henriques Abreu ; Joao Gama. IEEE, 2024. pp. 1-8 (Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023).

@inproceedings{cea6ea27d1ae4f708af2e2221afeab13,

title = "Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs",

abstract = "Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional Neural Networks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers results comparable to the state-of-the-art.",

keywords = "micro-expressions, emotion, GCN, Deep learning, graph networks, deep learning",

author = "Nikin Matharaarachchi and Pasha, \{Muhammad Fermi\} and Sonya Coleman and Dermot Kerr",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 22nd International Conference on Machine Learning and Applications, ICMLA'23 ; Conference date: 15-12-2023 Through 17-12-2023",

year = "2024",

month = mar,

day = "19",

doi = "10.1109/ICMLA58977.2023.00055",

language = "English",

isbn = "979-8-3503-1891-3",

series = "Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023",

publisher = "IEEE",

pages = "1--8",

editor = "\{Arif Wani\}, M. and Mihai Boicu and Moamar Sayed-Mouchaweh and Abreu, \{Pedro Henriques\} and Joao Gama",

booktitle = "2023 International Conference on Machine Learning and Applications (ICMLA)",

address = "United States",

url = "https://www.icmla-conference.org/icmla23/",

}

Matharaarachchi, N, Pasha, MF, Coleman, S & Kerr, D 2024, Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs. in M Arif Wani, M Boicu, M Sayed-Mouchaweh, PH Abreu & J Gama (eds), 2023 International Conference on Machine Learning and Applications (ICMLA). Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023, IEEE, pp. 1-8, 22nd International Conference on Machine Learning and Applications, Florida, United States, 15/12/23. https://doi.org/10.1109/ICMLA58977.2023.00055

Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs. / Matharaarachchi, Nikin; Pasha, Muhammad Fermi; Coleman, Sonya et al.
2023 International Conference on Machine Learning and Applications (ICMLA). ed. / M. Arif Wani; Mihai Boicu; Moamar Sayed-Mouchaweh; Pedro Henriques Abreu; Joao Gama. IEEE, 2024. p. 1-8 (Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs

AU - Matharaarachchi, Nikin

AU - Pasha, Muhammad Fermi

AU - Coleman, Sonya

AU - Kerr, Dermot

PY - 2024/3/19

Y1 - 2024/3/19

N2 - Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional Neural Networks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers results comparable to the state-of-the-art.

AB - Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional Neural Networks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers results comparable to the state-of-the-art.

KW - micro-expressions

KW - emotion

KW - GCN

KW - Deep learning

KW - graph networks

KW - deep learning

UR - https://www.icmla-conference.org/icmla23/

UR - https://www.scopus.com/pages/publications/85190114593

U2 - 10.1109/ICMLA58977.2023.00055

DO - 10.1109/ICMLA58977.2023.00055

M3 - Conference contribution

SN - 979-8-3503-1891-3

T3 - Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023

SP - 1

EP - 8

BT - 2023 International Conference on Machine Learning and Applications (ICMLA)

A2 - Arif Wani, M.

A2 - Boicu, Mihai

A2 - Sayed-Mouchaweh, Moamar

A2 - Abreu, Pedro Henriques

A2 - Gama, Joao

PB - IEEE

T2 - 22nd International Conference on Machine Learning and Applications

Y2 - 15 December 2023 through 17 December 2023

ER -

Matharaarachchi N, Pasha MF, Coleman S , Kerr D. Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs. In Arif Wani M, Boicu M, Sayed-Mouchaweh M, Abreu PH, Gama J, editors, 2023 International Conference on Machine Learning and Applications (ICMLA). IEEE. 2024. p. 1-8. (Proceedings - 22nd IEEE International Conference on Machine Learning and Applications, ICMLA 2023). Epub 2024 Mar 19. doi: 10.1109/ICMLA58977.2023.00055

Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs

Abstract

Publication series

Conference

Bibliographical note

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this