Semantics-Guided Object Removal for Facial Images: with Broad Applicability and Robust Style Preservation

09/29/2022
by   Jookyung Song, et al.
0

Object removal and image inpainting in facial images is a task in which objects that occlude a facial image are specifically targeted, removed, and replaced by a properly reconstructed facial image. Two different approaches utilizing U-net and modulated generator respectively have been widely endorsed for this task for their unique advantages but notwithstanding each method's innate disadvantages. U-net, a conventional approach for conditional GANs, retains fine details of unmasked regions but the style of the reconstructed image is inconsistent with the rest of the original image and only works robustly when the size of the occluding object is small enough. In contrast, the modulated generative approach can deal with a larger occluded area in an image and provides a more consistent style, yet it usually misses out on most of the detailed features. This trade-off between these two models necessitates an invention of a model that can be applied to any size of mask while maintaining a consistent style and preserving minute details of facial features. Here, we propose Semantics-Guided Inpainting Network (SGIN) which itself is a modification of the modulated generator, aiming to take advantage of its advanced generative capability and preserve the high-fidelity details of the original image. By using the guidance of a semantic map, our model is capable of manipulating facial features which grants direction to the one-to-many problem for further practicability.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

research
02/13/2022

Diverse facial inpainting guided by exemplars

Facial image inpainting is a task of filling visually realistic and sema...
research
05/09/2018

SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting

In this paper, we focus on image inpainting task, aiming at recovering t...
research
07/03/2019

Mask Embedding in conditional GAN for Guided Synthesis of High Resolution Images

Recent advancements in conditional Generative Adversarial Networks (cGAN...
research
01/18/2021

Iterative Facial Image Inpainting using Cyclic Reverse Generator

Facial image inpainting is a challenging problem as it requires generati...
research
01/26/2022

Interactive Image Inpainting Using Semantic Guidance

Image inpainting approaches have achieved significant progress with the ...
research
07/11/2022

Snow Mask Guided Adaptive Residual Network for Image Snow Removal

Image restoration under severe weather is a challenging task. Most of th...
research
06/14/2021

Pixel Sampling for Style Preserving Face Pose Editing

The existing auto-encoder based face pose editing methods primarily focu...

Please sign up or login with your details

Forgot password? Click here to reset