DeepAI
Log In Sign Up

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

03/08/2020
by   Sachit Menon, et al.
0

The primary aim of single-image super-resolution is to construct a high-resolution (HR) image from a corresponding low-resolution (LR) input. In previous approaches, which have generally been supervised, the training objective typically measures a pixel-wise average distance between the super-resolved (SR) and HR images. Optimizing such metrics often leads to blurring, especially in high variance (detailed) regions. We propose an alternative formulation of the super-resolution problem based on creating realistic SR images that downscale correctly. We present a novel super-resolution algorithm addressing this problem, PULSE (Photo Upsampling via Latent Space Exploration), which generates high-resolution, realistic images at resolutions previously unseen in the literature. It accomplishes this in an entirely self-supervised fashion and is not confined to a specific degradation operator used during training, unlike previous methods (which require training on databases of LR-HR image pairs for supervised learning). Instead of starting with the LR image and slowly adding detail, PULSE traverses the high-resolution natural image manifold, searching for images that downscale to the original LR image. This is formalized through the "downscaling loss," which guides exploration through the latent space of a generative model. By leveraging properties of high-dimensional Gaussians, we restrict the search space to guarantee that our outputs are realistic. PULSE thereby generates super-resolved images that both are realistic and downscale correctly. We show extensive experimental results demonstrating the efficacy of our approach in the domain of face super-resolution (also known as face hallucination). Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.

READ FULL TEXT

page 1

page 7

page 11

page 12

page 14

page 15

page 16

page 17

04/09/2020

DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution

Super-resolution (SR) is by definition ill-posed. There are infinitely m...
03/04/2020

Creating High Resolution Images with a Latent Adversarial Generator

Generating realistic images is difficult, and many formulations for this...
11/16/2021

A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Realistic hyperspectral image (HSI) super-resolution (SR) techniques aim...
11/30/2022

MrSARP: A Hierarchical Deep Generative Prior for SAR Image Super-resolution

Generative models learned from training using deep learning methods can ...
07/24/2022

Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network

Normalizing flow models have been used successfully for generative image...
11/05/2021

Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic Super-resolution

Super-resolution is an ill-posed problem, where a ground-truth high-reso...
07/27/2020

3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning

3D human shape and pose estimation from monocular images has been an act...