Pervasive Sound Sensing: A Weakly Supervised Training Approach

Daniel Kelly, Brian Caulfield

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Modern smartphones present an ideal device for pervasive sensing of human behavior. Microphones have the potential to reveal key information about a person's behavior. However, they have been utilized to a significantly lesser extent than other smartphone sensors in the context of human behavior sensing. We postulate that, in order for microphones to be useful in behavior sensing applications, the analysis techniques must be flexible and allow easy modification of the types of sounds to be sensed. A simplification of the training data collection process could allow a more flexible sound classification framework. We hypothesize that detailed training, a prerequisite for the majority of sound sensing techniques, is not necessary and that a significantly less detailed and time consuming data collection process can be carried out, allowing even a nonexpert to conduct the collection, labeling, and training process. To test this hypothesis, we implement a diverse density-based multiple instance learning framework, to identify a target sound, and a bag trimming algorithm, which, using the target sound, automatically segments weakly labeled sound clips to construct an accurate training set. Experiments reveal that our hypothesis is a valid one and results show that classifiers, trained using the automatically segmented training sets, were able to accurately classify unseen sound samples with accuracies comparable to supervised classifiers, achieving an average F-measure of 0.969 and 0.87 for two weakly supervised datasets.
LanguageEnglish
Pages123
Number of pages135
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume46
Issue number1
Early online date6 Feb 2015
DOIs
Publication statusPublished - 1 Jan 2016

Fingerprint

Acoustic waves
Smartphones
Microphones
Classifiers
Trimming
Labeling
Sensors
Experiments

Keywords

  • Diverse density (DD)
  • pattern recognition
  • pervasive sensing
  • sound classification
  • weak supervision

Cite this

@article{d6b18aa16b014edcb9eba268f63fafbb,
title = "Pervasive Sound Sensing: A Weakly Supervised Training Approach",
abstract = "Modern smartphones present an ideal device for pervasive sensing of human behavior. Microphones have the potential to reveal key information about a person's behavior. However, they have been utilized to a significantly lesser extent than other smartphone sensors in the context of human behavior sensing. We postulate that, in order for microphones to be useful in behavior sensing applications, the analysis techniques must be flexible and allow easy modification of the types of sounds to be sensed. A simplification of the training data collection process could allow a more flexible sound classification framework. We hypothesize that detailed training, a prerequisite for the majority of sound sensing techniques, is not necessary and that a significantly less detailed and time consuming data collection process can be carried out, allowing even a nonexpert to conduct the collection, labeling, and training process. To test this hypothesis, we implement a diverse density-based multiple instance learning framework, to identify a target sound, and a bag trimming algorithm, which, using the target sound, automatically segments weakly labeled sound clips to construct an accurate training set. Experiments reveal that our hypothesis is a valid one and results show that classifiers, trained using the automatically segmented training sets, were able to accurately classify unseen sound samples with accuracies comparable to supervised classifiers, achieving an average F-measure of 0.969 and 0.87 for two weakly supervised datasets.",
keywords = "Diverse density (DD), pattern recognition, pervasive sensing, sound classification, weak supervision",
author = "Daniel Kelly and Brian Caulfield",
year = "2016",
month = "1",
day = "1",
doi = "10.1109/TCYB.2015.2396291",
language = "English",
volume = "46",
pages = "123",
journal = "IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics",
issn = "1083-4419",
number = "1",

}

Pervasive Sound Sensing: A Weakly Supervised Training Approach. / Kelly, Daniel; Caulfield, Brian.

In: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 46, No. 1, 01.01.2016, p. 123.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Pervasive Sound Sensing: A Weakly Supervised Training Approach

AU - Kelly, Daniel

AU - Caulfield, Brian

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Modern smartphones present an ideal device for pervasive sensing of human behavior. Microphones have the potential to reveal key information about a person's behavior. However, they have been utilized to a significantly lesser extent than other smartphone sensors in the context of human behavior sensing. We postulate that, in order for microphones to be useful in behavior sensing applications, the analysis techniques must be flexible and allow easy modification of the types of sounds to be sensed. A simplification of the training data collection process could allow a more flexible sound classification framework. We hypothesize that detailed training, a prerequisite for the majority of sound sensing techniques, is not necessary and that a significantly less detailed and time consuming data collection process can be carried out, allowing even a nonexpert to conduct the collection, labeling, and training process. To test this hypothesis, we implement a diverse density-based multiple instance learning framework, to identify a target sound, and a bag trimming algorithm, which, using the target sound, automatically segments weakly labeled sound clips to construct an accurate training set. Experiments reveal that our hypothesis is a valid one and results show that classifiers, trained using the automatically segmented training sets, were able to accurately classify unseen sound samples with accuracies comparable to supervised classifiers, achieving an average F-measure of 0.969 and 0.87 for two weakly supervised datasets.

AB - Modern smartphones present an ideal device for pervasive sensing of human behavior. Microphones have the potential to reveal key information about a person's behavior. However, they have been utilized to a significantly lesser extent than other smartphone sensors in the context of human behavior sensing. We postulate that, in order for microphones to be useful in behavior sensing applications, the analysis techniques must be flexible and allow easy modification of the types of sounds to be sensed. A simplification of the training data collection process could allow a more flexible sound classification framework. We hypothesize that detailed training, a prerequisite for the majority of sound sensing techniques, is not necessary and that a significantly less detailed and time consuming data collection process can be carried out, allowing even a nonexpert to conduct the collection, labeling, and training process. To test this hypothesis, we implement a diverse density-based multiple instance learning framework, to identify a target sound, and a bag trimming algorithm, which, using the target sound, automatically segments weakly labeled sound clips to construct an accurate training set. Experiments reveal that our hypothesis is a valid one and results show that classifiers, trained using the automatically segmented training sets, were able to accurately classify unseen sound samples with accuracies comparable to supervised classifiers, achieving an average F-measure of 0.969 and 0.87 for two weakly supervised datasets.

KW - Diverse density (DD)

KW - pattern recognition

KW - pervasive sensing

KW - sound classification

KW - weak supervision

U2 - 10.1109/TCYB.2015.2396291

DO - 10.1109/TCYB.2015.2396291

M3 - Article

VL - 46

SP - 123

JO - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

T2 - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

JF - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

SN - 1083-4419

IS - 1

ER -