Improved Image Generation via Sparse Modeling

04/01/2021
by   Roy Ganz, et al.
0

The interest of the deep learning community in image synthesis has grown massively in recent years. Nowadays, deep generative methods, and especially Generative Adversarial Networks (GANs), are leading to state-of-the-art performance, capable of synthesizing images that appear realistic. While the efforts for improving the quality of the generated images are extensive, most attempts still consider the generator part as an uncorroborated "black-box". In this paper, we aim to provide a better understanding and design of the image generation process. We interpret existing generators as implicitly relying on sparsity-inspired models. More specifically, we show that generators can be viewed as manifestations of the Convolutional Sparse Coding (CSC) and its Multi-Layered version (ML-CSC) synthesis processes. We leverage this observation by explicitly enforcing a sparsifying regularization on appropriately chosen activation layers in the generator, and demonstrate that this leads to improved image synthesis. Furthermore, we show that the same rationale and benefits apply to generators serving inverse problems, demonstrated on the Deep Image Prior (DIP) method.

READ FULL TEXT

page 7

page 9

page 13

research
07/11/2018

Generative Adversarial Networks with Decoder-Encoder Output Noise

In recent years, research on image generation methods has been developin...
research
09/29/2020

TinyGAN: Distilling BigGAN for Conditional Image Generation

Generative Adversarial Networks (GANs) have become a powerful approach f...
research
05/07/2018

MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

Recently, generative adversarial networks (GANs) have shown promising pe...
research
11/27/2020

Image Generators with Conditionally-Independent Pixel Synthesis

Existing image generator networks rely heavily on spatial convolutions a...
research
12/10/2020

Slimmable Generative Adversarial Networks

Generative adversarial networks (GANs) have achieved remarkable progress...
research
04/20/2023

Using Text-to-Image Generation for Architectural Design Ideation

The recent progress of text-to-image generation has been recognized in a...
research
08/29/2023

Learning Modulated Transformation in GANs

The success of style-based generators largely benefits from style modula...

Please sign up or login with your details

Forgot password? Click here to reset