MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

by   David Keetae Park, et al.
UNC Charlotte
Korea University

Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.


page 3

page 5

page 6


Generating Images Part by Part with Composite Generative Adversarial Networks

Image generation remains a fundamental problem in artificial intelligenc...

Hierarchical Mixtures of Generators for Adversarial Learning

Generative adversarial networks (GANs) are deep neural networks that all...

Lessons Learned from the Training of GANs on Artificial Datasets

Generative Adversarial Networks (GANs) have made great progress in synth...

Improved Image Generation via Sparse Modeling

The interest of the deep learning community in image synthesis has grown...

Sparse Generative Adversarial Network

We propose a new approach to Generative Adversarial Networks (GANs) to a...

Selective Sampling and Mixture Models in Generative Adversarial Networks

In this paper, we propose a multi-generator extension to the adversarial...

Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

In this work, we propose a novel Cycle In Cycle Generative Adversarial N...

Please sign up or login with your details

Forgot password? Click here to reset