A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models

03/15/2012
by   Miika Pihlaja, et al.
0

We introduce a new family of estimators for unnormalized statistical models. Our family of estimators is parameterized by two nonlinear functions and uses a single sample from an auxiliary distribution, generalizing Maximum Likelihood Monte Carlo estimation of Geyer and Thompson (1992). The family is such that we can estimate the partition function like any other parameter in the model. The estimation is done by optimizing an algebraically simple, well defined objective function, which allows for the use of dedicated optimization methods. We establish consistency of the estimator family and give an expression for the asymptotic covariance matrix, which enables us to further analyze the influence of the nonlinearities and the auxiliary density on estimation performance. Some estimators in our family are particularly stable for a wide range of auxiliary densities. Interestingly, a specific choice of the nonlinearity establishes a connection between density estimation and classification by nonlinear logistic regression. Finally, the optimal amount of auxiliary samples relative to the given amount of the data is considered from the perspective of computational efficiency.

READ FULL TEXT
research
11/10/2020

A Statistical Perspective on Coreset Density Estimation

Coresets have emerged as a powerful tool to summarize data by selecting ...
research
05/21/2018

Weighted batch means estimators in Markov chain Monte Carlo

This paper proposes a family of weighted batch means variance estimators...
research
12/23/2019

An improper estimator with optimal excess risk in misspecified density estimation and logistic regression

We introduce a procedure for predictive conditional density estimation u...
research
06/13/2023

Learning Unnormalized Statistical Models via Compositional Optimization

Learning unnormalized statistical models (e.g., energy-based models) is ...
research
12/16/2019

Estimation of Auction Models with Shape Restrictions

We introduce several new estimation methods that leverage shape constrai...
research
05/10/2019

Selectivity Estimation with Deep Likelihood Models

Selectivity estimation has long been grounded in statistical tools for d...

Please sign up or login with your details

Forgot password? Click here to reset