Skip to main navigation Skip to search Skip to main content

Anisotropic Optical Flow Guided Adaptive Multi-Stage Video Inpainting

  • Northeastern University

Research output: Contribution to journal › Article › peer-review

157 Downloads (Pure)

Abstract

Video inpainting is attracting more attention due to the potential applications of video object removal and video content restoration. Current approaches either use end-to-end methods to generate missing pixels directly or perform indirect transfer for known regions based on motion field guidance. However, such approaches cannot handle both high-resolution images and diverse degrees of scene variation between adjacent video frames, and they cannot achieve clear and accurate inpainting effects for large continuous missing areas. To this end, we propose an adaptive multi-stage interval video inpainting algorithm guided by anisotropic optical flow. First, we customize an optical flow inpainting method guided by single image inpainting, enabling optical flow to maintain a strong self-healing ability over a large range of missing areas. Then, the interval mechanism adaptively determines the required temporal neighbors for missing pixels by assessing video attributes and inpainted optical flow results. After the missing pixels complete the multi-candidate information fusion in their associated temporal neighbors, we obtain spatio-temporally consistent and accurate results. Finally, extensive experiments on the YouTubeVOS, DAVIS, A2D2, and custom datasets show that our proposed approach has achieved state-of-the-art performance with good environmental migration ability.
Original languageEnglish
Pages (from-to)5643-5657
Number of pages15
JournalIEEE Transactions on Multimedia
Volume28
Early online date9 Mar 2026
DOIs
Publication statusPublished online - 9 Mar 2026

Bibliographical note

© 1999-2012 IEEE.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 61973066 and in part by the Major Science and Technology Projects of Liaoning Province under Grant 2021JH1/10400049.

FundersFunder number
National Natural Science Foundation of China61973066
2021JH1/10400049

    Keywords

    • Multi-stage video inpainting
    • adaptive interval mechanism
    • multi-candidate information fusion
    • optical flow inpainting

    Fingerprint

    Dive into the research topics of 'Anisotropic Optical Flow Guided Adaptive Multi-Stage Video Inpainting'. Together they form a unique fingerprint.

    Cite this