Dataset Condensation with Latent Space Knowledge Factorization and Sharing

08/21/2022
by   Hae Beom Lee, et al.
0

In this paper, we introduce a novel approach for systematically solving dataset condensation problem in an efficient manner by exploiting the regularity in a given dataset. Instead of condensing the dataset directly in the original input space, we assume a generative process of the dataset with a set of learnable codes defined in a compact latent space followed by a set of tiny decoders which maps them differently to the original input space. By combining different codes and decoders interchangeably, we can dramatically increase the number of synthetic examples with essentially the same parameter count, because the latent space is much lower dimensional and since we can assume as many decoders as necessary to capture different styles represented in the dataset with negligible cost. Such knowledge factorization allows efficient sharing of information between synthetic examples in a systematic way, providing far better trade-off between compression ratio and quality of the generated examples. We experimentally show that our method achieves new state-of-the-art records by significant margins on various benchmark datasets such as SVHN, CIFAR10, CIFAR100, and TinyImageNet.

READ FULL TEXT
research
10/29/2020

Latent Space Oddity: Exploring Latent Spaces to Design Guitar Timbres

We introduce a novel convolutional network architecture with an interpre...
research
02/11/2022

Multi-level Latent Space Structuring for Generative Control

Truncation is widely used in generative models for improving the quality...
research
12/21/2019

Deep Automodulators

We introduce a novel autoencoder model that deviates from traditional au...
research
04/30/2023

StyleGenes: Discrete and Efficient Latent Distributions for GANs

We propose a discrete latent distribution for Generative Adversarial Net...
research
05/12/2022

Overparameterization Improves StyleGAN Inversion

Deep generative models like StyleGAN hold the promise of semantic image ...
research
01/16/2023

Simplex Autoencoders

Synthetic data generation is increasingly important due to privacy conce...
research
06/29/2018

Sparse Three-parameter Restricted Indian Buffet Process for Understanding International Trade

This paper presents a Bayesian nonparametric latent feature model specia...

Please sign up or login with your details

Forgot password? Click here to reset