Density Estimation with Autoregressive Bayesian Predictives

06/13/2022
by   Sahra Ghalebikesabi, et al.
14

Bayesian methods are a popular choice for statistical inference in small-data regimes due to the regularization effect induced by the prior, which serves to counteract overfitting. In the context of density estimation, the standard Bayesian approach is to target the posterior predictive. In general, direct estimation of the posterior predictive is intractable and so methods typically resort to approximating the posterior distribution as an intermediate step. The recent development of recursive predictive copula updates, however, has made it possible to perform tractable predictive density estimation without the need for posterior approximation. Although these estimators are computationally appealing, they tend to struggle on non-smooth data distributions. This is largely due to the comparatively restrictive form of the likelihood models from which the proposed copula updates were derived. To address this shortcoming, we consider a Bayesian nonparametric model with an autoregressive likelihood decomposition and Gaussian process prior, which yields a data-dependent bandwidth parameter in the copula update. Further, we formulate a novel parameterization of the bandwidth using an autoregressive neural network that maps the data into a latent space, and is thus able to capture more complex dependencies in the data. Our extensions increase the modelling capacity of existing recursive Bayesian density estimators, achieving state-of-the-art results on tabular data sets.

READ FULL TEXT

page 2

page 5

page 20

page 30

page 31

research
03/29/2021

Martingale Posterior Distributions

The prior distribution on parameters of a likelihood is the usual starti...
research
02/20/2013

Estimating Continuous Distributions in Bayesian Classifiers

When modeling a probability distribution with a Bayesian network, we are...
research
02/12/2021

Robust and integrative Bayesian neural networks for likelihood-free parameter inference

State-of-the-art neural network-based methods for learning summary stati...
research
03/09/2011

A Kernel Approach to Tractable Bayesian Nonparametrics

Inference in popular nonparametric Bayesian models typically relies on s...
research
05/18/2018

Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows

We present Sequential Neural Likelihood (SNL), a new method for Bayesian...
research
11/05/2020

Hidden Markov Pólya trees for high-dimensional distributions

The Pólya tree (PT) process is a general-purpose Bayesian nonparametric ...
research
07/22/2019

Bayesian estimation of probabilistic sensitivity measures

Computer experiments are becoming increasingly important in scientific i...

Please sign up or login with your details

Forgot password? Click here to reset