Optimized Kernel Entropy Components

This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in Kernel Principal Components Analysis. In this work, we propose an extension of the KECA method, named Optimized KECA (OKECA), that directly extracts the optimal features retaining most of the data entropy by means of compacting the information in very few features (often in just one or two). The proposed method produces features which have higher expressive power. In particular, it is based on the Independent Component Analysis (ICA) framework, and introduces an extra rotation to the eigen-decomposition, which is optimized via gradient ascent search. This maximum entropy preservation suggests that OKECA features are more efficient than KECA features for density estimation. In addition, a critical issue in both methods is the selection of the kernel parameter since it critically affects the resulting performance. Here we analyze the most common kernel length-scale selection criteria. Results of both methods are illustrated in different synthetic and real problems. Results show that 1) OKECA returns projections with more expressive power than KECA, 2) the most successful rule for estimating the kernel parameter is based on maximum likelihood, and 3) OKECA is more robust to the selection of the length-scale parameter in kernel density estimation.

READ FULL TEXT

page 4

page 5

page 6

research
12/12/2012

Tree-dependent Component Analysis

We present a generalization of independent component analysis (ICA), whe...
research
05/14/2018

A One-Class Decision Tree Based on Kernel Density Estimation

One-Class Classification (OCC) is a domain of machine learning which ach...
research
02/04/2019

Numerical performance of Penalized Comparison to Overfitting for multivariate kernel density estimation

Kernel density estimation is a well known method involving a smoothing p...
research
04/18/2015

Fast optimization of Multithreshold Entropy Linear Classifier

Multithreshold Entropy Linear Classifier (MELC) is a density based model...
research
05/30/2020

A Spatio-Temporal Kernel Density Estimation Framework for Predictive Crime Hotspot Mapping and Evaluation

Predictive hotspot mapping plays a critical role in hotspot policing. Ex...
research
05/22/2023

Complexity measure, kernel density estimation, bandwidth selection, and the efficient market hypothesis

We are interested in the nonparametric estimation of the probability den...

Please sign up or login with your details

Forgot password? Click here to reset