Diversity-Generated Image Inpainting with Style Extraction

12/04/2019
by   Weiwei Cai, et al.
33

The latest methods based on deep learning have achieved amazing results regarding the complex work of inpainting large missing areas in an image. This type of method generally attempts to generate one single "optimal" inpainting result, ignoring many other plausible results. However, considering the uncertainty of the inpainting task, one sole result can hardly be regarded as a desired regeneration of the missing area. In view of this weakness, which is related to the design of the previous algorithms, we propose a novel deep generative model equipped with a brand new style extractor which can extract the style noise (a latent vector) from the ground truth image. Once obtained, the extracted style noise and the ground truth image are both input into the generator. We also craft a consistency loss that guides the generated image to approximate the ground truth. Meanwhile, the same extractor captures the style noise from the generated image, which is forced to approach the input noise according to the consistency loss. After iterations, our generator is able to learn the styles corresponding to multiple sets of noise. The proposed model can generate a (sufficiently large) number of inpainting results consistent with the context semantics of the image. Moreover, we check the effectiveness of our model on three databases, i.e., CelebA, Agricultural Disease, and MauFlex. Compared to state-of-the-art inpainting methods, this model is able to offer desirable inpainting results with both a better quality and higher diversity, for example, on the human face, we can even output the different gaze angles of the eyes and whether they have glasses, etc. The code and model will be made available on https://github.com/vivitsai/SEGAN.

READ FULL TEXT

page 2

page 9

page 10

page 14

page 15

page 16

research
09/02/2020

Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention

Deep learning (DL) has demonstrated its powerful capabilities in the fie...
research
07/26/2016

Semantic Image Inpainting with Deep Generative Models

Semantic image inpainting is a challenging task where large missing regi...
research
02/07/2020

Image Fine-grained Inpainting

Image inpainting techniques have shown promising improvement with the as...
research
05/29/2019

Coherent Semantic Attention for Image Inpainting

The latest deep learning-based approaches have shown promising results f...
research
07/16/2022

CNN-based Euler's Elastica Inpainting with Deep Energy and Deep Image Prior

Euler's elastica constitute an appealing variational image inpainting mo...
research
05/22/2023

EMEF: Ensemble Multi-Exposure Image Fusion

Although remarkable progress has been made in recent years, current mult...
research
06/29/2018

Outfit Generation and Style Extraction via Bidirectional LSTM and Autoencoder

When creating an outfit, style is a criterion in selecting each fashion ...

Please sign up or login with your details

Forgot password? Click here to reset