A Multi-Task Deep Learning Approach for Sensor-based Human Activity Recognition and Segmentation

Furong Duan, Tao Zhu, Jinqiang Wang, Liming Chen, Huansheng Ning, Yaping Wan

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)
45 Downloads (Pure)


Deep learning (DL) for sensor-based human activity recognition (HAR) has been a focus of research in recent years. Sensor data stream segmentation is a core element in HAR, which has currently been treated as an independent preprocessing task, usually with a fixed-size window. This has led to two critical problems, namely the multiclass window problem caused by possible multiple activities within a fixed-size window and the fluctuation of prediction results due to noisy data and oversegmentation. To address these research challenges, in this article, we conceive a novel multitask DL approach to segmenting and recognizing human activity simultaneously. Specifically, we propose a multiscale window method based on feature sequence generation to overcome the multiclass window problem. We develop a novel boundary offset prediction algorithm to adjust a window’s boundary to tackle the oversegmentation issue. In addition, we design a multitask framework to streamline and optimize the activity recognition and segmentation tasks simultaneously. We conduct extensive experiments on eight benchmark datasets to evaluate the proposed framework and associated methods. Initial results show that our approach outperforms the performance of current state-of-the-art HAR methods.
Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Instrumentation and Measurement
Publication statusPublished (in print/issue) - 16 May 2023

Bibliographical note

Publisher Copyright:
© 1963-2012 IEEE.


  • Task analysis
  • Multitasking
  • Feature extraction
  • Convolution
  • Deep learning
  • Prediction algorithms
  • Indexes
  • Activity recognition
  • activity segmentation
  • deep learning (DL)
  • multitask learning (MTL)
  • sensors


Dive into the research topics of 'A Multi-Task Deep Learning Approach for Sensor-based Human Activity Recognition and Segmentation'. Together they form a unique fingerprint.

Cite this