DeepAI
Log In Sign Up

SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data

03/29/2021
by   Jinwoo Kim, et al.
17

Generative modeling of set-structured data, such as point clouds, requires reasoning over local and global structures at various scales. However, adopting multi-scale frameworks for ordinary sequential data to a set-structured data is nontrivial as it should be invariant to the permutation of its elements. In this paper, we propose SetVAE, a hierarchical variational autoencoder for sets. Motivated by recent progress in set encoding, we build SetVAE upon attentive modules that first partition the set and project the partition back to the original cardinality. Exploiting this module, our hierarchical VAE learns latent variables at multiple scales, capturing coarse-to-fine dependency of the set elements while achieving permutation invariance. We evaluate our model on point cloud generation task and achieve competitive performance to the prior arts with substantially smaller model capacity. We qualitatively demonstrate that our model generalizes to unseen set sizes and learns interesting subset relations without supervision. Our implementation is available at https://github.com/jw9730/setvae.

READ FULL TEXT

page 15

page 16

page 17

page 18

03/08/2022

DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction

Existing permutation-invariant methods can be divided into two categorie...
10/12/2022

LION: Latent Point Diffusion Models for 3D Shape Generation

Denoising diffusion models (DDMs) have shown promising results in 3D poi...
09/26/2021

Fully Spiking Variational Autoencoder

Spiking neural networks (SNNs) can be run on neuromorphic devices with u...
07/31/2022

CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning

Processing 3D data efficiently has always been a challenge. Spatial oper...
10/13/2018

Point Cloud GAN

Generative Adversarial Networks (GAN) can achieve promising performance ...
06/26/2020

Conditional Set Generation with Transformers

A set is an unordered collection of unique elements–and yet many machine...

Code Repositories

setvae

[CVPR'21] SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data


view repo