TY - JOUR
T1 - uMoDT: an unobtrusive multi-occupant detection and tracking using robust Kalman filter for real-time activity recognition
AU - Razzaq, Muhammad Asif
AU - Quero, Javier Medina
AU - Cleland, I
AU - Nugent, CD
AU - Akhtar, Usman
AU - Bilal, Hafiz Syed Muhammad
AU - Ur Rehman, Ubaid
AU - Lee, Sungyoung
PY - 2020/10/31
Y1 - 2020/10/31
N2 - Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.
AB - Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.
KW - Classification
KW - Human activity recognition
KW - Image processing
KW - Object detection
KW - Tracking
UR - https://pure.ulster.ac.uk/en/publications/umodt-an-unobtrusive-multi-occupant-detection-and-tracking-using-
UR - http://www.scopus.com/inward/record.url?scp=85086787303&partnerID=8YFLogxK
UR - https://link.springer.com/article/10.1007/s00530-020-00664-7
U2 - 10.1007/s00530-020-00664-7
DO - 10.1007/s00530-020-00664-7
M3 - Article
SN - 0942-4962
VL - 26
SP - 553
EP - 569
JO - Multimedia Systems
JF - Multimedia Systems
IS - 5
ER -