uMoDT: An unobtrusive Multi-occupant Detection and Tracking using robust Kalman lter for real-time activity recognition

Muhammad Asif Razzaq, Javier Medina, I Cleland, CD Nugent, Usman Akhtar, Hafiz Syed Bilal Ali, Ubaid Ur Rehman, Sungyoung Lee

Research output: Contribution to journalArticle

Abstract

Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.

Original languageEnglish
Pages (from-to)553-569
Number of pages17
JournalMultimedia Systems
Volume26
Issue number5
Early online date23 Jun 2020
DOIs
Publication statusE-pub ahead of print - 23 Jun 2020

Keywords

  • Classification
  • Human activity recognition
  • Image processing
  • Object detection
  • Tracking

Fingerprint Dive into the research topics of 'uMoDT: An unobtrusive Multi-occupant Detection and Tracking using robust Kalman lter for real-time activity recognition'. Together they form a unique fingerprint.

  • Profiles

    Cite this