DeepAI
Log In Sign Up

Variational Gibbs inference for statistical model estimation from incomplete data

11/25/2021
by   Vaidotas Simkus, et al.
0

Statistical models are central to machine learning with broad applicability across a range of downstream tasks. The models are typically controlled by free parameters that are estimated from data by maximum-likelihood estimation. However, when faced with real-world datasets many of the models run into a critical issue: they are formulated in terms of fully-observed data, whereas in practice the datasets are plagued with missing data. The theory of statistical model estimation from incomplete data is conceptually similar to the estimation of latent-variable models, where powerful tools such as variational inference (VI) exist. However, in contrast to standard latent-variable models, parameter estimation with incomplete data often requires estimating exponentially-many conditional distributions of the missing variables, hence making standard VI methods intractable. We address this gap by introducing variational Gibbs inference (VGI), a new general-purpose method to estimate the parameters of statistical models from incomplete data. We validate VGI on a set of synthetic and real-world estimation tasks, estimating important machine learning models, VAEs and normalising flows, from incomplete data. The proposed method, whilst general-purpose, achieves competitive or better performance than existing model-specific estimation methods.

READ FULL TEXT

page 2

page 3

page 27

10/18/2018

Variational Noise-Contrastive Estimation

Unnormalised latent variable models are a broad and flexible class of st...
03/08/2016

Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling

We study parameter inference in large-scale latent variable models. We f...
01/16/2022

Reconstruction of Incomplete Wildfire Data using Deep Generative Models

We present our submission to the Extreme Value Analysis 2021 Data Challe...
11/13/2018

A General Method for Amortizing Variational Filtering

We introduce the variational filtering EM algorithm, a simple, general-p...
04/12/2020

A Machine Learning Approach for Flagging Incomplete Bid-rigging Cartels

We propose a new method for flagging bid rigging, which is particularly ...
04/06/2015

A New Approach to Building the Interindustry Input--Output Table

We present a new approach to estimating the interdependence of industrie...