Dataset shift is a major challenge in the non-stationary environments wherein the input data distribution may change over time. In a time-series data, detecting the dataset shift point, where the distribution changes its properties is of utmost interest. Dataset shift exists in a broad range of real-world systems. In such systems, there is a need for continuous monitoring of the process behavior and tracking the state of the shift so as to decide about initiating adaptive corrections in a timely manner. This paper presents a novel method to detect the shift-point based on a two-stage structure involving Exponentially WeightedMoving Average (EWMA) chart and Kolmogorov-Smirnov test, which substantially reduces type-I error rate. The algorithm is suitable to be run in real-time. Its performance is evaluated through experiments using synthetic and real-world datasets. Results show effectiveness of the proposed approach in terms of decreased type-I error and tolerable increase in detection time delay.
|Title of host publication||Artificial Intelligence Applications and Innovations|
|Editors||Harris Papadopoulos, Andreas. S. Andreou, Lazaros IIiadis, IIias Maglogiannis|
|Place of Publication||London|
|Publication status||Published - 1 Oct 2013|
Raza, H., Prasad, G., & Li, Y. (2013). EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments. In H. Papadopoulos, A. S. Andreou, L. IIiadis, & II. Maglogiannis (Eds.), Artificial Intelligence Applications and Innovations (Vol. 412, pp. 625-635). Springer. https://doi.org/10.1007/978-3-642-41142-7_63