uMoDT: an unobtrusive multi-occupant detection and tracking using robust Kalman filter for real-time activity recognition

Muhammad Asif Razzaq, Javier Medina Quero, I Cleland, CD Nugent, Usman Akhtar, Hafiz Syed Muhammad Bilal, Ubaid Ur Rehman, Sungyoung Lee

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)
95 Downloads (Pure)


Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.

Original languageEnglish
Pages (from-to)553-569
Number of pages17
JournalMultimedia Systems
Issue number5
Early online date23 Jun 2020
Publication statusPublished (in print/issue) - 31 Oct 2020


  • Classification
  • Human activity recognition
  • Image processing
  • Object detection
  • Tracking


Dive into the research topics of 'uMoDT: an unobtrusive multi-occupant detection and tracking using robust Kalman filter for real-time activity recognition'. Together they form a unique fingerprint.

Cite this