Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network

by   Min Zhang, et al.

Normalizing flow models have been used successfully for generative image super-resolution (SR) by approximating complex distribution of natural images to simple tractable distribution in latent space through Invertible Neural Networks (INN). These models can generate multiple realistic SR images from one low-resolution (LR) input using randomly sampled points in the latent space, simulating the ill-posed nature of image upscaling where multiple high-resolution (HR) images correspond to the same LR. Lately, the invertible process in INN has also been used successfully by bidirectional image rescaling models like IRN and HCFlow for joint optimization of downscaling and inverse upscaling, resulting in significant improvements in upscaled image quality. While they are optimized for image downscaling too, the ill-posed nature of image downscaling, where one HR image could be downsized to multiple LR images depending on different interpolation kernels and resampling methods, is not considered. A new downscaling latent variable, in addition to the original one representing uncertainties in image upscaling, is introduced to model variations in the image downscaling process. This dual latent variable enhancement is applicable to different image rescaling models and it is shown in extensive experiments that it can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.


page 6

page 8

page 9

page 10


Invertible Image Rescaling

High-resolution digital images are usually downscaled to fit various dis...

SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models

Single image super-resolution (SISR) aims to reconstruct high-resolution...

Learning a Mixture of Deep Networks for Single Image Super-Resolution

Single image super-resolution (SR) is an ill-posed problem which aims to...

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

The primary aim of single-image super-resolution is to construct a high-...

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Image Super-Resolution (SR) provides a promising technique to enhance th...

Towards Lightweight Super-Resolution with Dual Regression Learning

Deep neural networks have exhibited remarkable performance in image supe...

Perception-Distortion Trade-off in the SR Space Spanned by Flow Models

Flow-based generative super-resolution (SR) models learn to produce a di...