Beyond Smoothness: Incorporating Low-Rank Analysis into Nonparametric Density Estimation

by   Robert A. Vandermeulen, et al.

The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of O(1/√(n)) in L^1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/√(n) on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting



page 1

page 2

page 3

page 4


Nonparametric Estimation of Multi-View Latent Variable Models

Spectral methods have greatly advanced the estimation of latent variable...

Histogram Transform Ensembles for Density Estimation

We investigate an algorithm named histogram transform ensembles (HTE) de...

Learning HMMs with Nonparametric Emissions via Spectral Decompositions of Continuous Matrices

Recently, there has been a surge of interest in using spectral methods f...

Improving Nonparametric Density Estimation with Tensor Decompositions

While nonparametric density estimators often perform well on low dimensi...

L_1 density estimation from privatised data

We revisit the classical problem of nonparametric density estimation, bu...

Asymptotic Properties for Methods Combining Minimum Hellinger Distance Estimates and Bayesian Nonparametric Density Estimates

In frequentist inference, minimizing the Hellinger distance between a ke...

On the number of bins in a rank histogram

Rank histograms have become popular tools for assessing the reliability ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.