Posterior-Aided Regularization for Likelihood-Free Inference

02/15/2021
by   Dongjun Kim, et al.
1

The recent development of likelihood-free inference aims training a flexible density estimator for the target posterior with a set of input-output pairs from simulation. Given the diversity of simulation structures, it is difficult to find a single unified inference method for each simulation model. This paper proposes a universally applicable regularization technique, called Posterior-Aided Regularization (PAR), which is applicable to learning the density estimator, regardless of the model structure. Particularly, PAR solves the mode collapse problem that arises as the output dimension of the simulation increases. PAR resolves this posterior mode degeneracy through a mixture of 1) the reverse KL divergence with the mode seeking property; and 2) the mutual information for the high quality representation on likelihood. Because of the estimation intractability of PAR, we provide a unified estimation method of PAR to estimate both reverse KL term and mutual information term with a single neural network. Afterwards, we theoretically prove the asymptotic convergence of the regularized optimal solution to the unregularized optimal solution as the regularization magnitude converges to zero. Additionally, we empirically show that past sequential neural likelihood inferences in conjunction with PAR present the statistically significant gains on diverse simulation tasks.

READ FULL TEXT

page 1

page 17

page 20

page 22

research
10/15/2020

Sequential Likelihood-Free Inference with Implicit Surrogate Proposal

Bayesian inference without the access of likelihood, called likelihood-f...
research
06/03/2021

MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories

Simulation-based inference enables learning the parameters of a model ev...
research
05/18/2018

Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows

We present Sequential Neural Likelihood (SNL), a new method for Bayesian...
research
10/26/2022

Maximum Likelihood Learning of Energy-Based Models for Simulation-Based Inference

We introduce two synthetic likelihood methods for Simulation-Based Infer...
research
01/17/2022

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Diffusion probabilistic models (DPMs) represent a class of powerful gene...
research
11/16/2020

Regularized Mutual Information Neural Estimation

With the variational lower bound of mutual information (MI), the estimat...

Please sign up or login with your details

Forgot password? Click here to reset