Iterative Facial Image Inpainting using Cyclic Reverse Generator

01/18/2021
by   Yahya Dogan, et al.
11

Facial image inpainting is a challenging problem as it requires generating new pixels that include semantic information for masked key components in a face, e.g., eyes and nose. Recently, remarkable methods have been proposed in this field. Most of these approaches use encoder-decoder architectures and have different limitations such as allowing unique results for a given image and a particular mask. Alternatively, some approaches generate promising results using different masks with generator networks. However, these approaches are optimization-based and usually require quite a number of iterations. In this paper, we propose an efficient solution to the facial image painting problem using the Cyclic Reverse Generator (CRG) architecture, which provides an encoder-generator model. We use the encoder to embed a given image to the generator space and incrementally inpaint the masked regions until a plausible image is generated; a discriminator network is utilized to assess the generated images during the iterations. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. After the generation process, for the post processing, we utilize a Unet model that we trained specifically for this task to remedy the artifacts close to the mask boundaries. Our method allows applying sketch-based inpaintings, using variety of mask types, and producing multiple and diverse results. We qualitatively compared our method with the state-of-the-art models and observed that our method can compete with the other models in all mask types; it is particularly better in images where larger masks are utilized.

READ FULL TEXT

page 2

page 6

page 8

page 10

page 11

page 12

page 13

page 14

research
01/24/2022

RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Free-form inpainting is the task of adding new content to an image in th...
research
05/13/2023

AURA : Automatic Mask Generator using Randomized Input Sampling for Object Removal

The objective of the image inpainting task is to fill missing regions of...
research
07/03/2019

Mask Embedding in conditional GAN for Guided Synthesis of High Resolution Images

Recent advancements in conditional Generative Adversarial Networks (cGAN...
research
09/09/2018

Geometry-Aware Face Completion and Editing

Face completion is a challenging generation task because it requires gen...
research
09/29/2022

Semantics-Guided Object Removal for Facial Images: with Broad Applicability and Robust Style Preservation

Object removal and image inpainting in facial images is a task in which ...
research
11/30/2021

SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches

Sketch-based image manipulation is an interactive image editing task to ...
research
05/06/2019

The Missing Data Encoder: Cross-Channel Image Completion with Hide-And-Seek Adversarial Network

Image completion is the problem of generating whole images from fragment...

Please sign up or login with your details

Forgot password? Click here to reset