Context-Aware Image Inpainting with Learned Semantic Priors

06/14/2021
by   Wendong Zhang, et al.
0

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets.

READ FULL TEXT

page 1

page 3

page 5

page 6

12/08/2021

Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

Restoring reasonable and realistic content for arbitrary missing regions...
03/15/2020

Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes

Completing a corrupted image with correct structures and reasonable text...
08/25/2022

Unbiased Multi-Modality Guidance for Image Inpainting

Image inpainting is an ill-posed problem to recover missing or damaged i...
11/26/2017

Semantically Consistent Image Completion with Fine-grained Details

Image completion has achieved significant progress due to advances in ge...
11/05/2020

Hyperrealistic Image Inpainting with Hypergraphs

Image inpainting is a non-trivial task in computer vision due to multipl...
12/15/2020

Image Inpainting Guided by Coherence Priors of Semantics and Textures

Existing inpainting methods have achieved promising performance in recov...
02/11/2019

Semantic Hierarchical Priors for Intrinsic Image Decomposition

Intrinsic Image Decomposition (IID) is a challenging and interesting com...