Dataset Shift Detection in Non-stationary Environments Using EWMA Charts

Haider Raza, G Prasad, Yuhua Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Citations (Scopus)

Abstract

Dataset shift is a major challenge in the non-stationary environments wherein the input data distribution may change over time. Detecting the dataset shift point in the time-series data, where the distribution of time-series changes its properties, is of utmost interest. Dataset shift exists in a broad range of real-world systems. In such systems, there is a need for continuous monitoring of the process behavior and tracking the state of the shift so as to decide about initiating adaptive corrections in a timely manner. This paper presents an algorithm to detect the shift-point in a non-stationary time-series data. The proposed method detects the shift-point based on an exponentially weighted moving average (EWMA) control chart for auto-correlated observations. This algorithm is suitable to be run in real-time and monitors the data to detect the dataset shift. Its performance is evaluated through experiments using synthetic and real-world datasets. Results show that all the dataset-shifts are detected without the delay.
Original languageEnglish
Title of host publicationUnknown Host Publication
PublisherIEEE
Pages3151 -3156
Number of pages6
DOIs
Publication statusPublished - 13 Oct 2013
EventIEEE International Conference on Systems, Man, and Cybernetics - Manchester, UK.
Duration: 13 Oct 2013 → …

Conference

ConferenceIEEE International Conference on Systems, Man, and Cybernetics
Period13/10/13 → …

Fingerprint Dive into the research topics of 'Dataset Shift Detection in Non-stationary Environments Using EWMA Charts'. Together they form a unique fingerprint.

  • Cite this

    Raza, H., Prasad, G., & Li, Y. (2013). Dataset Shift Detection in Non-stationary Environments Using EWMA Charts. In Unknown Host Publication (pp. 3151 -3156). IEEE. https://doi.org/10.1109/SMC.2013.537