Pixel-wise Dense Detector for Image Inpainting

11/04/2020
by   Ruisong Zhang, et al.
0

Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar, which inevitably lose the position information of visual artifacts. Moreover, the adversarial loss and reconstruction loss (e.g., l1 loss) are combined with tradeoff weights, which are also difficult to tune. In this paper, we propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process. The generator follows an encoder-decoder architecture to fill the missing regions, and the detector using weakly supervised learning localizes the position of artifacts in a pixel-wise manner. Such position information makes the generator pay attention to artifacts and further enhance them. More importantly, we explicitly insert the output of the detector into the reconstruction loss with a weighting criterion, which balances the weight of the adversarial loss and reconstruction loss automatically rather than manual operation. Experiments on multiple public datasets show the superior performance of the proposed framework. The source code is available at https://github.com/Evergrow/GDN_Inpainting.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 4

page 6

page 7

page 8

page 9

page 10

page 11

09/02/2020

Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention

Deep learning (DL) has demonstrated its powerful capabilities in the fie...
11/25/2020

CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction

Recent deep generative inpainting methods use attention layers to allow ...
08/12/2020

Guided Collaborative Training for Pixel-wise Semi-Supervised Learning

We investigate the generalization of semi-supervised learning (SSL) to d...
11/10/2017

EddyNet: A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies

This work presents EddyNet, a deep learning based architecture for autom...
09/23/2019

Object Segmentation using Pixel-wise Adversarial Loss

Recent deep learning based approaches have shown remarkable success on o...
10/17/2021

Localization with Sampling-Argmax

Soft-argmax operation is commonly adopted in detection-based methods to ...
04/28/2021

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.