Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

by   Xingang Pan, et al.

Learning a good image prior is a long-term goal for image restoration and manipulation. While existing methods like deep image prior (DIP) capture low-level image statistics, there are still gaps toward an image prior that captures rich image semantics including color, spatial coherence, textures, and high-level concepts. This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. As shown in Fig.1, the deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images. It also enables diverse image manipulation including random jittering, image morphing, and category transfer. Such highly flexible restoration and manipulation are made possible through relaxing the assumption of existing GAN-inversion methods, which tend to fix the generator. Notably, we allow the generator to be fine-tuned on-the-fly in a progressive manner regularized by feature distance obtained by the discriminator in GAN. We show that these easy-to-implement and practical changes help preserve the reconstruction to remain in the manifold of nature image, and thus lead to more precise and faithful reconstruction for real images. Code is available at https://github.com/XingangPan/deepgenerative-prior.


page 17

page 18

page 19

page 20

page 21

page 22

page 23

page 24


Semantic-Aware Latent Space Exploration for Face Image Restoration

For image restoration, most existing deep learning based methods tend to...

QC-StyleGAN – Quality Controllable Image Generation and Manipulation

The introduction of high-quality image generation models, particularly t...

GAN Prior Embedded Network for Blind Face Restoration in the Wild

Blind face restoration (BFR) from severely degraded face images in the w...

Deep Image Prior

Deep convolutional networks have become a popular tool for image generat...

High-Resolution GAN Inversion for Degraded Images in Large Diverse Datasets

The last decades are marked by massive and diverse image data, which sho...

BigColor: Colorization using a Generative Color Prior for Natural Images

For realistic and vivid colorization, generative priors have recently be...

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Synthesizing visual content that meets users' needs often requires flexi...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset