Anisotropic Optical Flow Guided Adaptive Multi-Stage Video Inpainting

Video inpainting is attracting more attention due to the potential applications of video object removal and video content restoration. Current approaches either use end-to-end methods to generate missing pixels directly or perform indirect transfer for known regions based on motion field guidance. However, such approaches cannot handle both high-resolution images and diverse degrees of scene variation between adjacent video frames, and they cannot achieve clear and accurate inpainting effects for large continuous missing areas. To this end, we propose an adaptive multi-stage interval video inpainting algorithm guided by anisotropic optical flow. First, we customize an optical flow inpainting method guided by single image inpainting, enabling optical flow to maintain a strong self-healing ability over a large range of missing areas. Then, the interval mechanism adaptively determines the required temporal neighbors for missing pixels by assessing video attributes and inpainted optical flow results. After the missing pixels complete the multi-candidate information fusion in their associated temporal neighbors, we obtain spatio-temporally consistent and accurate results. Finally, extensive experiments on the YouTubeVOS, DAVIS, A2D2, and custom datasets show that our proposed approach has achieved state-of-the-art performance with good environmental migration ability.