EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments

Haider Raza, Girijesh Prasad, Yuhua Li

Research output: Chapter in Book/Report/Conference proceedingChapter

10 Citations (Scopus)

Abstract

Dataset shift is a major challenge in the non-stationary environments wherein the input data distribution may change over time. In a time-series data, detecting the dataset shift point, where the distribution changes its properties is of utmost interest. Dataset shift exists in a broad range of real-world systems. In such systems, there is a need for continuous monitoring of the process behavior and tracking the state of the shift so as to decide about initiating adaptive corrections in a timely manner. This paper presents a novel method to detect the shift-point based on a two-stage structure involving Exponentially WeightedMoving Average (EWMA) chart and Kolmogorov-Smirnov test, which substantially reduces type-I error rate. The algorithm is suitable to be run in real-time. Its performance is evaluated through experiments using synthetic and real-world datasets. Results show effectiveness of the proposed approach in terms of decreased type-I error and tolerable increase in detection time delay.
Original languageEnglish
Title of host publicationArtificial Intelligence Applications and Innovations
EditorsHarris Papadopoulos, Andreas. S. Andreou, Lazaros IIiadis, IIias Maglogiannis
Place of PublicationLondon
PublisherSpringer
Pages625-635
Volume412
ISBN (Print)978-3-642-41141-0
DOIs
Publication statusPublished - 1 Oct 2013

Fingerprint Dive into the research topics of 'EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments'. Together they form a unique fingerprint.

  • Cite this

    Raza, H., Prasad, G., & Li, Y. (2013). EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments. In H. Papadopoulos, A. S. Andreou, L. IIiadis, & II. Maglogiannis (Eds.), Artificial Intelligence Applications and Innovations (Vol. 412, pp. 625-635). London: Springer. https://doi.org/10.1007/978-3-642-41142-7_63