Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network

07/24/2022
by   Min Zhang, et al.
4

Normalizing flow models have been used successfully for generative image super-resolution (SR) by approximating complex distribution of natural images to simple tractable distribution in latent space through Invertible Neural Networks (INN). These models can generate multiple realistic SR images from one low-resolution (LR) input using randomly sampled points in the latent space, simulating the ill-posed nature of image upscaling where multiple high-resolution (HR) images correspond to the same LR. Lately, the invertible process in INN has also been used successfully by bidirectional image rescaling models like IRN and HCFlow for joint optimization of downscaling and inverse upscaling, resulting in significant improvements in upscaled image quality. While they are optimized for image downscaling too, the ill-posed nature of image downscaling, where one HR image could be downsized to multiple LR images depending on different interpolation kernels and resampling methods, is not considered. A new downscaling latent variable, in addition to the original one representing uncertainties in image upscaling, is introduced to model variations in the image downscaling process. This dual latent variable enhancement is applicable to different image rescaling models and it is shown in extensive experiments that it can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.

READ FULL TEXT

page 6

page 8

page 9

page 10

05/12/2020

Invertible Image Rescaling

High-resolution digital images are usually downscaled to fit various dis...
04/30/2021

SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models

Single image super-resolution (SISR) aims to reconstruct high-resolution...
01/03/2017

Learning a Mixture of Deep Networks for Single Image Super-Resolution

Single image super-resolution (SR) is an ill-posed problem which aims to...
03/08/2020

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

The primary aim of single-image super-resolution is to construct a high-...
12/07/2020

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Image Super-Resolution (SR) provides a promising technique to enhance th...
07/16/2022

Towards Lightweight Super-Resolution with Dual Regression Learning

Deep neural networks have exhibited remarkable performance in image supe...
09/18/2022

Perception-Distortion Trade-off in the SR Space Spanned by Flow Models

Flow-based generative super-resolution (SR) models learn to produce a di...