DeepAI AI Chat
Log In Sign Up

Validating Conditional Density Models and Bayesian Inference Algorithms

by   David Zhao, et al.

Conditional density models f(y|x), where x represents a potentially high-dimensional feature vector, are an integral part of uncertainty quantification in prediction and Bayesian inference. However, such models can be difficult to calibrate. While existing validation techniques can determine whether an approximated conditional density is compatible overall with a data sample, they lack practical procedures for identifying, localizing, and interpreting the nature of (statistically significant) discrepancies over the entire feature space. In this paper, we present more discerning diagnostics such as (i) the "Local Coverage Test" (LCT), which is able to distinguish an arbitrarily misspecified model from the true conditional density of the sample, and (ii) "Amortized Local P-P plots" (ALP), which can quickly provide interpretable graphical summaries of distributional differences at any location x in the feature space. Our validation procedures scale to high dimensions, and can potentially adapt to any type of data at hand. We demonstrate the effectiveness of LCT and ALP through a simulated experiment and a realistic application to parameter inference for galaxy images.


page 6

page 8


MD-split+: Practical Local Conformal Inference in High Dimensions

Quantifying uncertainty in model predictions is a common goal for practi...

Calibrated Predictive Distributions via Diagnostics for Conditional Coverage

Uncertainty quantification is crucial for assessing the predictive abili...

Asymptotically optimal inference in sparse sequence models with a simple data-dependent measure

For high-dimensional inference problems, statisticians have a number of ...

Bayesian Conditional Density Filtering

We propose a Conditional Density Filtering (C-DF) algorithm for efficien...

Model-based Bayesian inference of disease outbreak dynamics with invertible neural networks

Mathematical models in epidemiology strive to describe the dynamics and ...

Integrated organic inference (IOI): A reconciliation of statistical paradigms

It is recognised that the Bayesian approach to inference can not adequat...

GPU-Accelerated Hierarchical Bayesian Inference with Application to Modeling Cosmic Populations: CUDAHM

We describe a computational framework for hierarchical Bayesian inferenc...