DeepAI AI Chat
Log In Sign Up

PD-GAN: Probabilistic Diverse GAN for Image Inpainting

by   Hongyu Liu, et al.
City University of Hong Kong

We propose PD-GAN, a probabilistic diverse GAN for image inpainting. Given an input image with arbitrary hole regions, PD-GAN produces multiple inpainting results with diverse and visually realistic content. Our PD-GAN is built upon a vanilla GAN which generates images based on random noise. During image generation, we modulate deep features of input random noise from coarse-to-fine by injecting an initially restored image and the hole regions in multiple scales. We argue that during hole filling, the pixels near the hole boundary should be more deterministic (i.e., with higher probability trusting the context and initially restored image to create natural inpainting boundary), while those pixels lie in the center of the hole should enjoy more degrees of freedom (i.e., more likely to depend on the random noise for enhancing diversity). To this end, we propose spatially probabilistic diversity normalization (SPDNorm) inside the modulation to model the probability of generating a pixel conditioned on the context information. SPDNorm dynamically balances the realism and diversity inside the hole region, making the generated content more diverse towards the hole center and resemble neighboring image content more towards the hole boundary. Meanwhile, we propose a perceptual diversity loss to further empower PD-GAN for diverse content generation. Experiments on benchmark datasets including CelebA-HQ, Places2 and Paris Street View indicate that PD-GAN is effective for diverse and visually realistic image restoration.


page 1

page 5

page 6

page 7

page 8


Probabilistic Semantic Inpainting with Pixel Constrained CNNs

Semantic inpainting is the task of inferring missing pixels in an image ...

Diverse facial inpainting guided by exemplars

Facial image inpainting is a task of filling visually realistic and sema...

An Analysis of Generative Methods for Multiple Image Inpainting

Image inpainting refers to the restoration of an image with missing regi...

Guided Image Inpainting: Replacing an Image Region by Pulling Content from Another Image

Deep generative models have shown success in automatically synthesizing ...

Light-weight pixel context encoders for image inpainting

In this work we propose Pixel Content Encoders (PCE), a light-weight ima...

Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE

Given an incomplete image without additional constraint, image inpaintin...

CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training

Recent image inpainting methods have made great progress but often strug...