The Missing Data Encoder: Cross-Channel Image Completion with Hide-And-Seek Adversarial Network

05/06/2019
by   Arnaud Dapogny, et al.
0

Image completion is the problem of generating whole images from fragments only. It encompasses inpainting (generating a patch given its surrounding), reverse inpainting/extrapolation (generating the periphery given the central patch) as well as colorization (generating one or several channels given other ones). In this paper, we employ a deep network to perform image completion, with adversarial training as well as perceptual and completion losses, and call it the "missing data encoder" (MDE). We consider several configurations based on how the seed fragments are chosen. We show that training MDE for "random extrapolation and colorization" (MDE-REC), i.e. using random channel-independent fragments, allows a better capture of the image semantics and geometry. MDE training makes use of a novel "hide-and-seek" adversarial loss, where the discriminator seeks the original non-masked regions, while the generator tries to hide them. We validate our models both qualitatively and quantitatively on several datasets, showing their interest for image completion, unsupervised representation learning as well as face occlusion handling.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

page 8

research
01/01/2019

EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning

Over the last few years, deep learning techniques have yielded significa...
research
03/23/2023

Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning

This paper presents a new adversarial training framework for image inpai...
research
04/24/2023

GRIG: Few-Shot Generative Residual Image Inpainting

Image inpainting is the task of filling in missing or masked region of a...
research
03/27/2018

Structural inpainting

Scene-agnostic visual inpainting remains very challenging despite progre...
research
11/26/2017

Semantically Consistent Image Completion with Fine-grained Details

Image completion has achieved significant progress due to advances in ge...
research
01/18/2021

Iterative Facial Image Inpainting using Cyclic Reverse Generator

Facial image inpainting is a challenging problem as it requires generati...
research
12/16/2022

Free-form 3D Scene Inpainting with Dual-stream GAN

Nowadays, the need for user editing in a 3D scene has rapidly increased ...

Please sign up or login with your details

Forgot password? Click here to reset