An IoT management framework for decision making in instances of unreliability

Abstract

The Internet of Things (IoT) is a growing technology used in various critical real-world applications. However, several sources of uncertainty in IoT systems can lead to inaccurate decision-making, posing severe societal risks. Human Activity Recognition(HAR) and Intrusion Response (IR) are critical domains within the IoT where high-risk decisions are made. HAR aims to identify human activities, and IR aims to mitigate cybersecurity attacks. These domains pose challenges since accurate decision making requires a thorough understanding of the situation.

The thesis proposes a domain-agnostic IoT decision-making framework that integrates data analytics and domain knowledge to manage uncertainties and convert raw data into meaningful actions. The Model Calibration Module of the framework extracts and quantifies information, combines it with action rules, and performs cost-sensitive decision-making. The proposed Calibrated Random Forest (CRF) data analytic approach was evaluated in a Smart Home scenario and improved average precision, recall, and F1-Score from 74% to 76%, 75% to 95%, 88%, and 90%,respectively. The framework also identified 53.38% of instances of activities as reliable, 42.59% as transitional, and 4.03% as noisy.

The CRF model has improved the calibration of the Machine Learning (ML) model by43% in terms of Log Loss, 18% in terms of Brier Score, and 69% in terms of Expected Calibration Error (ECE) for an efficient Intrusion Response System (IRS). The cost sensitive ML-based IR prioritization mechanism has achieved a minimum cost of 134 with an accuracy of 99.90%. The Fuzzy Analytical Hierarchical Process (FAHP) has resulted in an increased accuracy of 5.84%, recall of 8.07%, and F1-Score of 7.59%for assigning actions by a human expert in the IRS. The event prioritization methodology incorporating domain knowledge has led to generating a labelled IR dataset combining expert knowledge and CRF output. The thesis concludes by identifying limitations and future directions.

Date of Award	Mar 2023
Original language	English
Sponsors	BT Ireland Innovation Centre (BTIIC)
Supervisor	Christopher Nugent (Supervisor), Jun Liu (Supervisor) & Adrian Moore (Supervisor)