DeepAI AI Chat
Log In Sign Up

Dimensionality reduction to maximize prediction generalization capability

by   Takuya Isomura, et al.

This work develops an analytically solvable unsupervised learning scheme that extracts the most informative components for predicting future inputs, termed predictive principal component analysis (PredPCA). Our scheme can effectively remove unpredictable observation noise and globally minimize the test prediction error. Mathematical analyses demonstrate that, with sufficiently high-dimensional observations that are generated by a linear or nonlinear system, PredPCA can identify the optimal hidden state representation, true system parameters, and true hidden state dimensionality, with a global convergence guarantee. We demonstrate the performance of PredPCA by using sequential visual inputs comprising hand-digits, rotating 3D objects, and natural scenes. It reliably and accurately estimates distinct hidden states and predicts future outcomes of previously unseen test input data, even in the presence of considerable observation noise. The simple model structure and low computational cost of PredPCA make it highly desirable as a learning scheme for biological neural networks and neuromorphic chips.


page 19

page 21


Dimensionality Reduction Ensembles

Ensemble learning has had many successes in supervised learning, but it ...

Unsupervised Discovery of Temporal Structure in Noisy Data with Dynamical Components Analysis

Linear dimensionality reduction methods are commonly used to extract low...

Quadratic speedup of global search using a biased crossover of two good solutions

The minimisation of cost functions is crucial in various optimisation fi...

On the achievability of blind source separation for high-dimensional nonlinear source mixtures

For many years, a combination of principal component analysis (PCA) and ...

Regression-based music emotion prediction using triplet neural networks

In this paper, we adapt triplet neural networks (TNNs) to a regression t...

Regression of high dimensional angular momentum states of light

The Orbital Angular Momentum (OAM) of light is an infinite-dimensional d...