Context-Aware Image Inpainting with Learned Semantic Priors

by   Wendong Zhang, et al.

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets.


page 1

page 3

page 5

page 6


Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

Restoring reasonable and realistic content for arbitrary missing regions...

Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes

Completing a corrupted image with correct structures and reasonable text...

Unbiased Multi-Modality Guidance for Image Inpainting

Image inpainting is an ill-posed problem to recover missing or damaged i...

Semantically Consistent Image Completion with Fine-grained Details

Image completion has achieved significant progress due to advances in ge...

Hyperrealistic Image Inpainting with Hypergraphs

Image inpainting is a non-trivial task in computer vision due to multipl...

Image Inpainting Guided by Coherence Priors of Semantics and Textures

Existing inpainting methods have achieved promising performance in recov...

Semantic Hierarchical Priors for Intrinsic Image Decomposition

Intrinsic Image Decomposition (IID) is a challenging and interesting com...