Beyond Smoothness: Incorporating Low-Rank Analysis into Nonparametric Density Estimation

04/02/2022
by   Robert A. Vandermeulen, et al.
0

The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of O(1/√(n)) in L^1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/√(n) on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

11/13/2013

Nonparametric Estimation of Multi-View Latent Variable Models

Spectral methods have greatly advanced the estimation of latent variable...
11/24/2019

Histogram Transform Ensembles for Density Estimation

We investigate an algorithm named histogram transform ensembles (HTE) de...
09/21/2016

Learning HMMs with Nonparametric Emissions via Spectral Decompositions of Continuous Matrices

Recently, there has been a surge of interest in using spectral methods f...
10/06/2020

Improving Nonparametric Density Estimation with Tensor Decompositions

While nonparametric density estimators often perform well on low dimensi...
07/27/2021

L_1 density estimation from privatised data

We revisit the classical problem of nonparametric density estimation, bu...
10/18/2018

Asymptotic Properties for Methods Combining Minimum Hellinger Distance Estimates and Bayesian Nonparametric Density Estimates

In frequentist inference, minimizing the Hellinger distance between a ke...
05/18/2020

On the number of bins in a rank histogram

Rank histograms have become popular tools for assessing the reliability ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.