Sampling constrained probability distributions using Spherical Augmentation

06/19/2015
by   Shiwei Lan, et al.
0

Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained domains could be quite challenging for commonly used sampling algorithms. In this paper, we propose a novel augmentation technique that handles a wide range of constraints by mapping the constrained domain to a sphere in the augmented space. By moving freely on the surface of this sphere, sampling algorithms handle constraints implicitly and generate proposals that remain within boundaries when mapped back to the original space. Our proposed method, called Spherical Augmentation, provides a mathematically natural and computationally efficient framework for sampling from constrained probability distributions. We show the advantages of our method over state-of-the-art sampling algorithms, such as exact Hamiltonian Monte Carlo, using several examples including truncated Gaussian distributions, Bayesian Lasso, Bayesian bridge regression, reconstruction of quantized stationary Gaussian process, and LDA for topic modeling.

READ FULL TEXT
research
09/26/2022

Sampling Constrained Continuous Probability Distributions: A Review

The problem of sampling constrained continuous distributions has frequen...
research
09/08/2017

Roll-back Hamiltonian Monte Carlo

We propose a new framework for Hamiltonian Monte Carlo (HMC) on truncate...
research
10/21/2021

An Efficient Scheme for Sampling in Constrained Domains

The creation of optimal samplers can be a challenging task, especially i...
research
01/04/2018

Bayesian Constraint Relaxation

Prior information often takes the form of parameter constraints. Bayesia...
research
06/28/2019

Fast and Exact Simulation of Multivariate Normal and Wishart Random Variables with Box Constraints

Models which include domain constraints occur in myriad contexts such as...
research
08/04/2020

The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation

Latent Dirichlet allocation (LDA) obtains essential information from dat...
research
09/24/2018

Flexible Mixture Modeling on Constrained Spaces

This paper addresses challenges in flexibly modeling multimodal data tha...

Please sign up or login with your details

Forgot password? Click here to reset