Structure First Detail Next: Image Inpainting with Pyramid Generator

06/16/2021
by   Shuyi Qu, et al.
0

Recent deep generative models have achieved promising performance in image inpainting. However, it is still very challenging for a neural network to generate realistic image details and textures, due to its inherent spectral bias. By our understanding of how artists work, we suggest to adopt a `structure first detail next' workflow for image inpainting. To this end, we propose to build a Pyramid Generator by stacking several sub-generators, where lower-layer sub-generators focus on restoring image structures while the higher-layer sub-generators emphasize image details. Given an input image, it will be gradually restored by going through the entire pyramid in a bottom-up fashion. Particularly, our approach has a learning scheme of progressively increasing hole size, which allows it to restore large-hole images. In addition, our method could fully exploit the benefits of learning with high-resolution images, and hence is suitable for high-resolution image inpainting. Extensive experimental results on benchmark datasets have validated the effectiveness of our approach compared with state-of-the-art methods.

READ FULL TEXT

page 3

page 5

page 6

page 8

research
12/04/2020

Generator Pyramid for High-Resolution Image Inpainting

Inpainting high-resolution images with large holes challenges existing d...
research
09/30/2020

Texture-aware Multi-resolution Image Inpainting

Recent GAN-based inpainting methods have shown remarkable performance us...
research
02/11/2020

Learning to Incorporate Structure Knowledge for Image Inpainting

This paper develops a multi-task learning framework that attempts to inc...
research
12/08/2021

Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

Restoring reasonable and realistic content for arbitrary missing regions...
research
05/24/2020

High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling

Existing image inpainting methods often produce artifacts when dealing w...
research
10/28/2022

Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering

Bokeh effect highlights an object (or any part of the image) while blurr...
research
04/03/2021

Aggregated Contextual Transformations for High-Resolution Image Inpainting

State-of-the-art image inpainting approaches can suffer from generating ...

Please sign up or login with your details

Forgot password? Click here to reset