How well does your sampler really work?

12/16/2017
by   Ryan Turner, et al.
0

We present a new data-driven benchmark system to evaluate the performance of new MCMC samplers. Taking inspiration from the COCO benchmark in optimization, we view this task as having critical importance to machine learning and statistics given the rate at which new samplers are proposed. The common hand-crafted examples to test new samplers are unsatisfactory; we take a meta-learning-like approach to generate benchmark examples from a large corpus of data sets and models. Surrogates of posteriors found in real problems are created using highly flexible density models including modern neural network based approaches. We provide new insights into the real effective sample size of various samplers per unit time and the estimation efficiency of the samplers per sample. Additionally, we provide a meta-analysis to assess the predictive utility of various MCMC diagnostics and perform a nonparametric regression to combine them.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/08/2018

Meta-Learning: A Survey

Meta-learning, or learning to learn, is the science of systematically ob...
research
12/03/2022

Meta Learning for Few-Shot Medical Text Classification

Medical professionals frequently work in a data constrained setting to p...
research
10/07/2021

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

Recent research has established the effectiveness of machine learning fo...
research
06/12/2018

Meta-Learning for Stochastic Gradient MCMC

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increa...
research
06/09/2022

Data-Efficient Brain Connectome Analysis via Multi-Task Meta-Learning

Brain networks characterize complex connectivities among brain regions a...
research
10/31/2022

A new benchmark for group distribution shifts in hand grasp regression for object manipulation. Can meta-learning raise the bar?

Understanding hand-object pose with computer vision opens the door to ne...
research
08/17/2019

Parametric Majorization for Data-Driven Energy Minimization Methods

Energy minimization methods are a classical tool in a multitude of compu...

Please sign up or login with your details

Forgot password? Click here to reset