DeepAI AI Chat
Log In Sign Up

Minimizing the Expected Posterior Entropy Yields Optimal Summary Statistics

by   Till Hoffmann, et al.
Harvard University

Extracting low-dimensional summary statistics from large datasets is essential for efficient (likelihood-free) inference. We propose obtaining summary statistics by minimizing the expected posterior entropy (EPE) under the prior predictive distribution of the model. We show that minimizing the EPE is equivalent to learning a conditional density estimator for the posterior as well as other information-theoretic approaches. Further summary extraction methods (including minimizing the L^2 Bayes risk, maximizing the Fisher information, and model selection approaches) are special or limiting cases of EPE minimization. We demonstrate that the approach yields high fidelity summary statistics by applying it to both a synthetic benchmark as well as a population genetics problem. We not only offer concrete recommendations for practitioners but also provide a unifying perspective for obtaining informative summary statistics.


Improving the Accuracy of Marginal Approximations in Likelihood-Free Inference via Localisation

Likelihood-free methods are an essential tool for performing inference f...

Decoupling Shrinkage and Selection in Gaussian Linear Factor Analysis

Factor Analysis is a popular method for modeling dependence in multivari...

Robust and integrative Bayesian neural networks for likelihood-free parameter inference

State-of-the-art neural network-based methods for learning summary stati...

Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network

Approximate Bayesian Computation (ABC) methods are used to approximate p...

On Model Selection with Summary Statistics

Recently, many authors have cast doubts on the validity of ABC model cho...

A Comparison of Likelihood-Free Methods With and Without Summary Statistics

Likelihood-free methods are useful for parameter estimation of complex m...

Summary Statistics for Partitionings and Feature Allocations

Infinite mixture models are commonly used for clustering. One can sample...

Code Repositories


Comparison of summary statistic selection methods with a unifying perspective.

view repo