Full Encoder: Make Autoencoders Learn Like PCA

03/25/2021
by   Zhouzheng Li, et al.
0

While the beta-VAE family is aiming to find disentangled representations and acquire human-interpretable generative factors, like what an ICA does in the linear domain, we propose Full Encoder: a novel unified autoencoder framework as a correspondence to PCA in the non-linear domain. The idea is to train an autoencoder with one latent variable first, then involve more latent variables progressively to refine the reconstruction results. The latent variables acquired with Full Encoder is stable and robust, as they always learn the same representation regardless the network initial states. Full Encoder can be used to determine the degrees of freedom in a non-linear system, and is useful for data compression or anomaly detection. Full Encoder can also be combined with beta-VAE framework to sort out the importance of the generative factors, providing more insights for non-linear system analysis. We created a toy dataset with a non-linear system to test the Full Encoder and compare its results to VAE and beta-VAE's results.

READ FULL TEXT

page 16

page 18

research
03/25/2023

Beta-VAE has 2 Behaviors: PCA or ICA?

Beta-VAE is a very classical model for disentangled representation learn...
research
12/17/2018

Variational Autoencoders Pursue PCA Directions (by Accident)

The Variational Autoencoder (VAE) is a powerful architecture capable of ...
research
10/12/2019

Disentangling Interpretable Generative Parameters of Random and Real-World Graphs

While a wide range of interpretable generative procedures for graphs exi...
research
06/27/2019

Tuning-Free Disentanglement via Projection

In representation learning and non-linear dimension reduction, there is ...
research
02/22/2021

Non-linear, Sparse Dimensionality Reduction via Path Lasso Penalized Autoencoders

High-dimensional data sets are often analyzed and explored via the const...
research
04/24/2021

Anomaly Detection for Solder Joints Using β-VAE

In the assembly process of printed circuit boards (PCB), most of the err...
research
11/26/2019

A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics

Disentangled encoding is an important step towards a better representati...

Please sign up or login with your details

Forgot password? Click here to reset