Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone

Federico Cruciani, I Cleland, CD Nugent, P McCullagh, Synnes Kare, Hallberg Josef

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such
datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate
the robustness of common supervised classification approaches under instances of noisy data.
We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generated
labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also
demonstrated how the presence of label noise could lower the f-score up to 64–74% depending on the
classification approach (Nearest Centroid and Multi-Class Support Vector Machine).
LanguageEnglish
Article number2203
JournalSensors
DOIs
Publication statusPublished - 9 Jul 2018

Fingerprint

annotations
Smartphones
Human Activities
availability
Labels
Availability
machine learning
ground truth
centroids
Labeling
marking
Support vector machines
Learning systems
Social Conditions
Neural networks
Noise
Smartphone
Datasets

Cite this

@article{b32bf27fb4ab413f967e0f270d03857f,
title = "Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone",
abstract = "Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of suchdatasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigatethe robustness of common supervised classification approaches under instances of noisy data.We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85{\%} average precision rate. Results obtained also show how a supervised approach trained using automatically generatedlabels achieved an 84{\%} f-score (using Neural Networks and Random Forests); however, results alsodemonstrated how the presence of label noise could lower the f-score up to 64–74{\%} depending on theclassification approach (Nearest Centroid and Multi-Class Support Vector Machine).",
author = "Federico Cruciani and I Cleland and CD Nugent and P McCullagh and Synnes Kare and Hallberg Josef",
year = "2018",
month = "7",
day = "9",
doi = "10.3390/s18072203",
language = "English",
journal = "Sensors",
issn = "1424-8220",
publisher = "MDPI",

}

Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone. / Cruciani, Federico; Cleland, I; Nugent, CD; McCullagh, P; Kare, Synnes; Josef, Hallberg.

In: Sensors, 09.07.2018.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone

AU - Cruciani, Federico

AU - Cleland, I

AU - Nugent, CD

AU - McCullagh, P

AU - Kare, Synnes

AU - Josef, Hallberg

PY - 2018/7/9

Y1 - 2018/7/9

N2 - Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of suchdatasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigatethe robustness of common supervised classification approaches under instances of noisy data.We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generatedlabels achieved an 84% f-score (using Neural Networks and Random Forests); however, results alsodemonstrated how the presence of label noise could lower the f-score up to 64–74% depending on theclassification approach (Nearest Centroid and Multi-Class Support Vector Machine).

AB - Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of suchdatasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigatethe robustness of common supervised classification approaches under instances of noisy data.We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generatedlabels achieved an 84% f-score (using Neural Networks and Random Forests); however, results alsodemonstrated how the presence of label noise could lower the f-score up to 64–74% depending on theclassification approach (Nearest Centroid and Multi-Class Support Vector Machine).

U2 - 10.3390/s18072203

DO - 10.3390/s18072203

M3 - Article

JO - Sensors

T2 - Sensors

JF - Sensors

SN - 1424-8220

M1 - 2203

ER -