Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders

01/19/2022
by   Sara Björk, et al.
0

Full characterization of the spectral behavior of generative models based on neural networks remains an open issue. Recent research has focused heavily on generative adversarial networks and the high-frequency discrepancies between real and generated images. The current solution to avoid this is to either replace transposed convolutions with bilinear up-sampling or add a spectral regularization term in the generator. It is well known that Variational Autoencoders (VAEs) also suffer from these issues. In this work, we propose a simple 2D Fourier transform-based spectral regularization loss for the VAE and show that it can achieve results equal to, or better than, the current state-of-the-art in frequency-aware losses for generative models. In addition, we experiment with altering the up-sampling procedure in the generator network and investigate how it influences the spectral performance of the model. We include experiments on synthetic and real data sets to demonstrate our results.

READ FULL TEXT
research
03/03/2020

Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions

Generative convolutional deep neural networks, e.g. popular GAN architec...
research
11/15/2019

Fourier Spectrum Discrepancies in Deep Network Generated Images

Advancements in deep generative models such as generative adversarial ne...
research
03/27/2021

Continuous Conditional Generative Adversarial Networks (cGAN) with Generator Regularization

Conditional Generative Adversarial Networks are known to be difficult to...
research
11/03/2021

On the Frequency Bias of Generative Models

The key objective of Generative Adversarial Networks (GANs) is to genera...
research
06/19/2023

Probabilistic matching of real and generated data statistics in generative adversarial networks

Generative adversarial networks constitute a powerful approach to genera...
research
03/31/2021

A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detection

CNN-based generative modelling has evolved to produce synthetic images i...
research
04/12/2023

Explicitly Minimizing the Blur Error of Variational Autoencoders

Variational autoencoders (VAEs) are powerful generative modelling method...

Please sign up or login with your details

Forgot password? Click here to reset