Pixel-wise Dense Detector for Image Inpainting

by   Ruisong Zhang, et al.

Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar, which inevitably lose the position information of visual artifacts. Moreover, the adversarial loss and reconstruction loss (e.g., l1 loss) are combined with tradeoff weights, which are also difficult to tune. In this paper, we propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process. The generator follows an encoder-decoder architecture to fill the missing regions, and the detector using weakly supervised learning localizes the position of artifacts in a pixel-wise manner. Such position information makes the generator pay attention to artifacts and further enhance them. More importantly, we explicitly insert the output of the detector into the reconstruction loss with a weighting criterion, which balances the weight of the adversarial loss and reconstruction loss automatically rather than manual operation. Experiments on multiple public datasets show the superior performance of the proposed framework. The source code is available at https://github.com/Evergrow/GDN_Inpainting.



There are no comments yet.


page 1

page 4

page 6

page 7

page 8

page 9

page 10

page 11


Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention

Deep learning (DL) has demonstrated its powerful capabilities in the fie...

CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction

Recent deep generative inpainting methods use attention layers to allow ...

Guided Collaborative Training for Pixel-wise Semi-Supervised Learning

We investigate the generalization of semi-supervised learning (SSL) to d...

EddyNet: A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies

This work presents EddyNet, a deep learning based architecture for autom...

Object Segmentation using Pixel-wise Adversarial Loss

Recent deep learning based approaches have shown remarkable success on o...

Localization with Sampling-Argmax

Soft-argmax operation is commonly adopted in detection-based methods to ...

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.