Variational Gibbs inference for statistical model estimation from incomplete data

11/25/2021
by   Vaidotas Simkus, et al.
0

Statistical models are central to machine learning with broad applicability across a range of downstream tasks. The models are typically controlled by free parameters that are estimated from data by maximum-likelihood estimation. However, when faced with real-world datasets many of the models run into a critical issue: they are formulated in terms of fully-observed data, whereas in practice the datasets are plagued with missing data. The theory of statistical model estimation from incomplete data is conceptually similar to the estimation of latent-variable models, where powerful tools such as variational inference (VI) exist. However, in contrast to standard latent-variable models, parameter estimation with incomplete data often requires estimating exponentially-many conditional distributions of the missing variables, hence making standard VI methods intractable. We address this gap by introducing variational Gibbs inference (VGI), a new general-purpose method to estimate the parameters of statistical models from incomplete data. We validate VGI on a set of synthetic and real-world estimation tasks, estimating important machine learning models, VAEs and normalising flows, from incomplete data. The proposed method, whilst general-purpose, achieves competitive or better performance than existing model-specific estimation methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 3

page 27

10/18/2018

Variational Noise-Contrastive Estimation

Unnormalised latent variable models are a broad and flexible class of st...
03/08/2016

Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling

We study parameter inference in large-scale latent variable models. We f...
01/16/2022

Reconstruction of Incomplete Wildfire Data using Deep Generative Models

We present our submission to the Extreme Value Analysis 2021 Data Challe...
11/13/2018

A General Method for Amortizing Variational Filtering

We introduce the variational filtering EM algorithm, a simple, general-p...
04/12/2020

A Machine Learning Approach for Flagging Incomplete Bid-rigging Cartels

We propose a new method for flagging bid rigging, which is particularly ...
04/06/2015

A New Approach to Building the Interindustry Input--Output Table

We present a new approach to estimating the interdependence of industrie...
09/11/2018

Solving Non-identifiable Latent Feature Models

Latent feature models (LFM)s are widely employed for extracting latent s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.