Video Snow Removal Based on Self-adaptation Snow Detection and Patch-based Gaussian Mixture Model

Bin Yang, Z Jia, Jie Yang, Nikola Kasabov

Research output: Contribution to journalArticle

Abstract

Video desnowing has become a challenging research topic in computer vision in recent years. Existing methods cannot remove most of the snow in heavy snow scenes and will cause the deformation of moving objects when used for snowy videos that include moving objects. These methods have poor generalizability, exhibiting poor performance when removing snow from videos with different resolutions. In this paper, we propose a new video snow removal method based on self-adaptation snow detection and a patch-based Gaussian mixture model (VSRSG). First, an optical flow estimation method and a support vector machine (SVM) are used to detect snowflakes, and a self-adaptation threshold process is used to remove dense snowflakes in the snowflake detection map to obtain a sparse snowflake detection map. Then, a patch-based Gaussian mixture model (PBGMM), which can remove moving objects and both sparse and dense snowflakes from videos and restore a clear video background, is applied for background modeling. A Markov random field (MRF) and self-adaptation threshold processing are used to extract sparse snowflakes and moving objects and combine them with the background to form an input video without dense snowflakes. Finally, a similar block matching method is employed to fill in the detected snowflake pixels with the information from adjacent frames to remove the sparse snowflakes in the near range. This method can also remove snowflakes in front of moving objects. Experiments show that the proposed method can simultaneously remove sparse snowflakes, dense snowflakes and snowflakes in front of moving objects and outperforms other state-of-the-art methods.
Original languageEnglish
Pages (from-to)1-14
Number of pages14
JournalIEEE Access
Early online date31 Aug 2020
DOIs
Publication statusE-pub ahead of print - 31 Aug 2020

Keywords

  • Snow
  • Filtering algorithms
  • Gaussian mixture model
  • Object detection
  • Rain

Fingerprint Dive into the research topics of 'Video Snow Removal Based on Self-adaptation Snow Detection and Patch-based Gaussian Mixture Model'. Together they form a unique fingerprint.

  • Cite this