PEPSI++: Fast and Lightweight Network for Image Inpainting

05/22/2019
by   Yong-Goo Shin, et al.
0

Generative adversarial network (GAN)-based image inpainting methods which utilize coarse-to-fine network with a contextual attention module (CAM) have shown remarkable performance. However, they require numerous computational resources such as convolution operations and network parameters due to two stacked generative networks, which results in a low speed. To address this problem, we propose a novel network structure called PEPSI: parallel extended-decoder path for semantic inpainting network, which aims at not only reducing hardware costs but also improving the inpainting performance. The PEPSI consists of a single shared encoding network and parallel decoding networks with coarse and inpainting paths. The coarse path generates a preliminary inpainting result to train the encoding network for prediction of features for the CAM. At the same time, the inpainting path results in higher inpainting quality with refined features reconstructed using the CAM. In addition, we propose a Diet-PEPSI which significantly reduces the network parameters while maintaining the performance. In the proposed method, we present a Diet-PEPSI unit (DPU) which effectively aggregates the global contextual information with a small number of parameters. Extensive experiments and comparisons with state-of-the-art image inpainting methods demonstrate that both PEPSI and Diet-PEPSI achieve significant improvements in qualitative scores and reduced computation cost.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

page 10

page 12

research
01/26/2022

Interactive Image Inpainting Using Semantic Guidance

Image inpainting approaches have achieved significant progress with the ...
research
04/06/2022

Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

The new alternative is to use deep learning to inpaint any image by util...
research
02/13/2023

Learning to Scale Temperature in Masked Self-Attention for Image Inpainting

Recent advances in deep generative adversarial networks (GAN) and self-a...
research
07/15/2023

Both Spatial and Frequency Cues Contribute to High-Fidelity Image Inpainting

Deep generative approaches have obtained great success in image inpainti...
research
02/11/2022

A Wasserstein GAN for Joint Learning of Inpainting and its Spatial Optimisation

Classic image inpainting is a restoration method that reconstructs missi...
research
01/01/2022

Adaptive Image Inpainting

Image inpainting methods have shown significant improvements by using de...
research
07/21/2020

Enhancement of damaged-image prediction through Cahn-Hilliard Image Inpainting

We assess the benefit of including an image inpainting filter before pas...

Please sign up or login with your details

Forgot password? Click here to reset