Detecting and Responding to Concept Drift in Business Processes

Lingkai Yang, Sally I McClean, MP Donnelly, Kevin Burke, Kashaf Khan

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
35 Downloads (Pure)

Abstract

Concept drift, which refers to changes in the underlying process structure or customer behaviour over time, is inevitable in business processes, causing challenges in ensuring that the learned model is a proper representation of the new data. Due to factors such as seasonal effects and policy updates, concept drifts can occur in customer transitions and time spent throughout the process, either suddenly or gradually. In a concept drift context, we can discard the old data and retrain the model using new observations (sudden drift) or combine the old data with the new data to update the model (gradual drift) or maintain the model as unchanged (no drift). In this paper, we model a response to concept drift as a sequential decision making problem by combing a hierarchical Markov model and a Markov decision process (MDP). The approach can detect concept drift, retrain the model and update customer profiles automatically. We validate the proposed approach on 68 artificial datasets and a real-world hospital billing dataset, with experimental results showing promising performance.
Original languageEnglish
Article number174
Number of pages19
JournalAlgorithms
Volume15
Issue number5
DOIs
Publication statusPublished (in print/issue) - 21 May 2022

Bibliographical note

Funding Information:
Acknowledgments: This research is supported by BTIIC (the British Telecom Ireland Innovation Centre), funded by British Telecom and Invest Northern Ireland.

Publisher Copyright:
© 2022 by the authors. Licensee MDPI, Basel, Switzerland.

Keywords

  • business process
  • concept drift
  • duration drift
  • hierarchical Markov model
  • Markov decision process

Fingerprint

Dive into the research topics of 'Detecting and Responding to Concept Drift in Business Processes'. Together they form a unique fingerprint.

Cite this