A Multi-Task Deep Learning Approach for Sensor-based Human Activity Recognition and Segmentation

Deep learning (DL) for sensor-based human activity recognition (HAR) has been a focus of research in recent years. Sensor data stream segmentation is a core element in HAR, which has currently been treated as an independent preprocessing task, usually with a fixed-size window. This has led to two critical problems, namely the multiclass window problem caused by possible multiple activities within a fixed-size window and the fluctuation of prediction results due to noisy data and oversegmentation. To address these research challenges, in this article, we conceive a novel multitask DL approach to segmenting and recognizing human activity simultaneously. Specifically, we propose a multiscale window method based on feature sequence generation to overcome the multiclass window problem. We develop a novel boundary offset prediction algorithm to adjust a window’s boundary to tackle the oversegmentation issue. In addition, we design a multitask framework to streamline and optimize the activity recognition and segmentation tasks simultaneously. We conduct extensive experiments on eight benchmark datasets to evaluate the proposed framework and associated methods. Initial results show that our approach outperforms the performance of current state-of-the-art HAR methods.