Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

by   Wendong Zhang, et al.
Shanghai Jiao Tong University

Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task. Although recent image inpainting models have made significant progress in generating vivid visual details, they can still lead to texture blurring or structural distortions due to contextual ambiguity when dealing with more complex scenes. To address this issue, we propose the Semantic Pyramid Network (SPN) motivated by the idea that learning multi-scale semantic priors from specific pretext tasks can greatly benefit the recovery of locally missing content in images. SPN consists of two components. First, it distills semantic priors from a pretext model into a multi-scale feature pyramid, achieving a consistent understanding of the global context and local structures. Within the prior learner, we present an optional module for variational inference to realize probabilistic image inpainting driven by various learned priors. The second component of SPN is a fully context-aware image generator, which adaptively and progressively refines low-level visual representations at multiple scales with the (stochastic) prior pyramid. We train the prior learner and the image generator as a unified model without any post-processing. Our approach achieves the state of the art on multiple datasets, including Places2, Paris StreetView, CelebA, and CelebA-HQ, under both deterministic and probabilistic inpainting setups.


page 1

page 4

page 8

page 9

page 11


Context-Aware Image Inpainting with Learned Semantic Priors

Recent advances in image inpainting have shown impressive results for ge...

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

High-quality image inpainting requires filling missing regions in a dama...

Unbiased Multi-Modality Guidance for Image Inpainting

Image inpainting is an ill-posed problem to recover missing or damaged i...

Image Inpainting via Generative Multi-column Convolutional Neural Networks

In this paper, we propose a generative multi-column network for image in...

Structure First Detail Next: Image Inpainting with Pyramid Generator

Recent deep generative models have achieved promising performance in ima...

Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes

Completing a corrupted image with correct structures and reasonable text...

Wavelet Prior Attention Learning in Axial Inpainting Network

Image inpainting is the task of filling masked or unknown regions of an ...

Code Repositories


Code for Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

view repo

Please sign up or login with your details

Forgot password? Click here to reset