Few-Shot Human Activity Recognition using lightweight Language Models

The lack of labeled data and model generalization
abilities have historically represented major obstacles for Human
Activity Recognition (HAR). Few-Shot Learning (FSL) addresses
both of these issues. However, its application to HAR, is particularly
difficult, since transferring models into new environments
requires the model to adapt not only to a new set of activity
labels but also, in many cases, to a different sensor configuration
and possibly even a different sensor modality. The ability of
Large Language Models (LLMs) to understand and interpret
natural language, together with their versatility as few-shot
learners, can greatly simplify model transfer. This is particularly
the case where sensor activations can be transformed into text
descriptions, since such descriptions abstract from the specific
sensor modality and configuration. Unfortunately, the application
of LLMs at the edge is typically hindered by their hardware
requirements, making it unfeasible or highly inefficient to deploy
these models to be used locally within smart environments. In
this context, we propose a lightweight LLM-based FSL approach
to facilitate model transfer with only a few labeled data samples,
while using a model small enough to be deployable at the
edge. Our results show that a relatively small BERT-based LLM
architecture (hundreds of millions of parameters vs. hundreds of
billions of parameters) can outperform larger models (including
Chat-GPT). The approach was evaluated on two challenging
datasets, namely the “van Kasteren (VK)” and the “VK houses”
datasets. Using our 10-shot FSL approach, we obtain a macro
average F-score of 52.75% on “VK” vs. a baseline F-score of
26.63% using Chat-GPT. On the “VK houses” dataset, our macro
average F-score is 36.02%, in contrast to 15.46% for Chat-GPT.