Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

12/08/2021
by   Wendong Zhang, et al.
3

Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task. Although recent image inpainting models have made significant progress in generating vivid visual details, they can still lead to texture blurring or structural distortions due to contextual ambiguity when dealing with more complex scenes. To address this issue, we propose the Semantic Pyramid Network (SPN) motivated by the idea that learning multi-scale semantic priors from specific pretext tasks can greatly benefit the recovery of locally missing content in images. SPN consists of two components. First, it distills semantic priors from a pretext model into a multi-scale feature pyramid, achieving a consistent understanding of the global context and local structures. Within the prior learner, we present an optional module for variational inference to realize probabilistic image inpainting driven by various learned priors. The second component of SPN is a fully context-aware image generator, which adaptively and progressively refines low-level visual representations at multiple scales with the (stochastic) prior pyramid. We train the prior learner and the image generator as a unified model without any post-processing. Our approach achieves the state of the art on multiple datasets, including Places2, Paris StreetView, CelebA, and CelebA-HQ, under both deterministic and probabilistic inpainting setups.

READ FULL TEXT

page 1

page 4

page 8

page 9

page 11

research
06/14/2021

Context-Aware Image Inpainting with Learned Semantic Priors

Recent advances in image inpainting have shown impressive results for ge...
research
04/16/2019

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

High-quality image inpainting requires filling missing regions in a dama...
research
08/25/2022

Unbiased Multi-Modality Guidance for Image Inpainting

Image inpainting is an ill-posed problem to recover missing or damaged i...
research
10/20/2018

Image Inpainting via Generative Multi-column Convolutional Neural Networks

In this paper, we propose a generative multi-column network for image in...
research
06/16/2021

Structure First Detail Next: Image Inpainting with Pyramid Generator

Recent deep generative models have achieved promising performance in ima...
research
03/15/2020

Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes

Completing a corrupted image with correct structures and reasonable text...
research
06/07/2022

Wavelet Prior Attention Learning in Axial Inpainting Network

Image inpainting is the task of filling masked or unknown regions of an ...

Please sign up or login with your details

Forgot password? Click here to reset