
PCAAE: Principal Component Analysis Autoencoder for organising the latent space of generative networks
Autoencoders and generative models produce some of the most spectacular ...
read it

Deep Learning for CaseBased Reasoning through Prototypes: A Neural Network that Explains Its Predictions
Deep neural networks are widely used for classification. These deep mode...
read it

RateDistortion Optimization Guided Autoencoder for Generative Approach with quantitatively measurable latent space
In the generative model approach of machine learning, it is essential to...
read it

Autoencoder Node Saliency: Selecting Relevant Latent Representations
The autoencoder is an artificial neural network model that learns hidden...
read it

Symbolic Pregression: Discovering Physical Laws from Raw Distorted Video
We present a method for unsupervised learning of equations of motion for...
read it

Image Semantic Transformation: Faster, Lighter and Stronger
We propose ImageSemanticTransformationReconstructionCircle(ISTRC) mo...
read it

Conditioning Autoencoder Latent Spaces for RealTime Timbre Interpolation and Synthesis
We compare standard autoencoder topologies' performances for timbre gene...
read it
A PCAlike Autoencoder
An autoencoder is a neural network which data projects to and from a lower dimensional latent space, where this data is easier to understand and model. The autoencoder consists of two subnetworks, the encoder and the decoder, which carry out these transformations. The neural network is trained such that the output is as close to the input as possible, the data having gone through an information bottleneck : the latent space. This tool bears significant ressemblance to Principal Component Analysis (PCA), with two main differences. Firstly, the autoencoder is a nonlinear transformation, contrary to PCA, which makes the autoencoder more flexible and powerful. Secondly, the axes found by a PCA are orthogonal, and are ordered in terms of the amount of variability which the data presents along these axes. This makes the interpretability of the PCA much greater than that of the autoencoder, which does not have these attributes. Ideally, then, we would like an autoencoder whose latent space consists of independent components, ordered by decreasing importance to the data. In this paper, we propose an algorithm to create such a network. We create an iterative algorithm which progressively increases the size of the latent space, learning a new dimension at each step. Secondly, we propose a covariance loss term to add to the standard autoencoder loss function, as well as a normalisation layer just before the latent space, which encourages the latent space components to be statistically independent. We demonstrate the results of this autoencoder on simple geometric shapes, and find that the algorithm indeed finds a meaningful representation in the latent space. This means that subsequent interpolation in the latent space has meaning with respect to the geometric properties of the images.
READ FULL TEXT
Comments
There are no comments yet.