Supervised Vector Quantized Variational Autoencoder for Learning Interpretable Global Representations

09/24/2019
by   Yifan Xue, et al.
25

Learning interpretable representations of data remains a central challenge in deep learning. When training a deep generative model, the observed data are often associated with certain categorical labels, and, in parallel with learning to regenerate data and simulate new data, learning an interpretable representation of each class of data is also a process of acquiring knowledge. Here, we present a novel generative model, referred to as the Supervised Vector Quantized Variational AutoEncoder (S-VQ-VAE), which combines the power of supervised and unsupervised learning to obtain a unique, interpretable global representation for each class of data. Compared with conventional generative models, our model has three key advantages: first, it is an integrative model that can simultaneously learn a feature representation for individual data point and a global representation for each class of data; second, the learning of global representations with embedding codes is guided by supervised information, which clearly defines the interpretation of each code; and third, the global representations capture crucial characteristics of different classes, which reveal similarity and differences of statistical structures underlying different groups of data. We evaluated the utility of S-VQ-VAE on a machine learning benchmark dataset, the MNIST dataset, and on gene expression data from the Library of Integrated Network-Based Cellular Signatures (LINCS). We proved that S-VQ-VAE was able to learn the global genetic characteristics of samples perturbed by the same class of perturbagen (PCL), and further revealed the mechanism correlations between PCLs. Such knowledge is crucial for promoting new drug development for complex diseases like cancer.

READ FULL TEXT

page 7

page 8

page 9

research
12/15/2020

Unsupervised Learning of Global Factors in Deep Generative Models

We present a novel deep generative model based on non i.i.d. variational...
research
07/20/2022

Cancer Subtyping by Improved Transcriptomic Features Using Vector Quantized Variational Autoencoder

Defining and separating cancer subtypes is essential for facilitating pe...
research
06/07/2016

Towards a Neural Statistician

An efficient learner is one who reuses what they already know to tackle ...
research
06/26/2017

Dr.VAE: Drug Response Variational Autoencoder

We present two deep generative models based on Variational Autoencoders ...
research
11/08/2016

Variational Lossy Autoencoder

Representation learning seeks to expose certain aspects of observed data...
research
02/04/2022

Robust Vector Quantized-Variational Autoencoder

Image generative models can learn the distributions of the training data...
research
05/29/2023

Autoencoding Conditional Neural Processes for Representation Learning

Conditional neural processes (CNPs) are a flexible and efficient family ...

Please sign up or login with your details

Forgot password? Click here to reset