DeepAI AI Chat
Log In Sign Up

Scaling Nonparametric Bayesian Inference via Subsample-Annealing

by   Fritz Obermeyer, et al.

We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse temperature beta(t) in an annealing schedule. Gibbs sampling at high temperature (i.e., with a very small subsample) can more quickly explore sketches of the final latent state by (a) making longer jumps around latent space (as in block Gibbs) and (b) lowering energy barriers (as in simulated annealing). We prove subsample annealing speeds up mixing time N^2 -> N in a simple clustering model and exp(N) -> N in another class of models, where N is data size. Empirically subsample-annealing outperforms naive Gibbs sampling in accuracy-per-wallclock time, and can scale to larger datasets and deeper hierarchical models. We demonstrate improved inference on million-row subsamples of US Census data and network log data and a 307-row hospital rating dataset, using a Pitman-Yor generalization of the Cross Categorization model.


page 1

page 2

page 3

page 4


Variational Tempering

Variational inference (VI) combined with data subsampling enables approx...

A Simulated Annealing Approach to Bayesian Inference

A generic algorithm for the extraction of probabilistic (Bayesian) infor...

High-quality Thermal Gibbs Sampling with Quantum Annealing Hardware

Quantum Annealing (QA) was originally intended for accelerating the solu...

Selective Inference for Latent Block Models

Model selection in latent block models has been a challenging but import...

Unified Software Design Patterns for Simulated Annealing

Any optimization algorithm programming interface can be seen as a black-...