Conditional Density Estimation with Dimensionality Reduction via Squared-Loss Conditional Entropy Minimization

by   Voot Tangkaratt, et al.

Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional density itself is preferable, but conditional density estimation (CDE) is challenging in high-dimensional space. A naive approach to coping with high-dimensionality is to first perform dimensionality reduction (DR) and then execute CDE. However, such a two-step process does not perform well in practice because the error incurred in the first DR step can be magnified in the second CDE step. In this paper, we propose a novel single-shot procedure that performs CDE and DR simultaneously in an integrated way. Our key idea is to formulate DR as the problem of minimizing a squared-loss variant of conditional entropy, and this is solved via CDE. Thus, an additional CDE step is not needed after DR. We demonstrate the usefulness of the proposed method through extensive experiments on various datasets including humanoid robot transition and computer art.


Transformer-based dimensionality reduction

Recently, Transformer is much popular and plays an important role in the...

Multi-point dimensionality reduction to improve projection layout reliability

In ordinary Dimensionality Reduction (DR), each data instance in an m-di...

Local Explanation of Dimensionality Reduction

Dimensionality reduction (DR) is a popular method for preparing and anal...

A Visual Analytics Framework for Reviewing Multivariate Time-Series Data with Dimensionality Reduction

Data-driven problem solving in many real-world applications involves ana...

bigMap: Big Data Mapping with Parallelized t-SNE

We introduce an improved unsupervised clustering protocol specially suit...

A Confident Information First Principle for Parametric Reduction and Model Selection of Boltzmann Machines

Typical dimensionality reduction (DR) methods are often data-oriented, f...

Dimensionality Reduction has Quantifiable Imperfections: Two Geometric Bounds

In this paper, we investigate Dimensionality reduction (DR) maps in an i...

Please sign up or login with your details

Forgot password? Click here to reset