DeepAI
Log In Sign Up

Deep Dimension Reduction for Supervised Representation Learning

06/10/2020
by   Jian Huang, et al.
37

The success of deep supervised learning depends on its automatic data representation abilities. Among all the characteristics of an ideal representation for high-dimensional complex data, information preservation, low dimensionality and disentanglement are the most essential ones. In this work, we propose a deep dimension reduction (DDR) approach to achieving a good data representation with these characteristics for supervised learning. At the population level, we formulate the ideal representation learning task as finding a nonlinear dimension reduction map that minimizes the sum of losses characterizing conditional independence and disentanglement. We estimate the target map at the sample level nonparametrically with deep neural networks. We derive a bound on the excess risk of the deep nonparametric estimator. The proposed method is validated via comprehensive numerical experiments and real data analysis in the context of regression and classification.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/21/2022

Deep Sufficient Representation Learning via Mutual Information

We propose a mutual information-based sufficient representation learning...
06/03/2019

Transformed Central Quantile Subspace

We present a dimension reduction technique for the conditional quantiles...
06/19/2021

Deep Learning for Functional Data Analysis with Adaptive Basis Layers

Despite their widespread success, the application of deep neural network...
10/24/2010

Local Component Analysis for Nonparametric Bayes Classifier

The decision boundaries of Bayes classifier are optimal because they lea...
09/02/2011

Gradient-based kernel dimension reduction for supervised learning

This paper proposes a novel kernel approach to linear dimension reductio...
02/20/2021

Nonlinear dimension reduction for surrogate modeling using gradient information

We introduce a method for the nonlinear dimension reduction of a high-di...