A moment-matching metric for latent variable generative models

10/04/2021
by   Cédric Beaulac, et al.
0

It can be difficult to assess the quality of a fitted model when facing unsupervised learning problems. Latent variable models, such as variation autoencoders and Gaussian mixture models, are often trained with likelihood-based approaches. In scope of Goodhart's law, when a metric becomes a target it ceases to be a good metric and therefore we should not use likelihood to assess the quality of the fit of these models. The solution we propose is a new metric for model comparison or regularization that relies on moments. The concept is to study the difference between the data moments and the model moments using a matrix norm, such as the Frobenius norm. We show how to use this new metric for model comparison and then for regularization. It is common to draw samples from the fitted distribution when evaluating latent variable models and we show that our proposed metric is faster to compute and has a smaller variance that this alternative. We conclude this article with a proof of concept of both applications and we discuss future work.

READ FULL TEXT
research
02/13/2018

Leveraging the Exact Likelihood of Deep Latent Variable Models

Deep latent variable models combine the approximation abilities of deep ...
research
06/12/2018

Gaussian mixture models with Wasserstein distance

Generative models with both discrete and continuous latent variables are...
research
02/13/2018

Leveraging the Exact Likelihood of Deep Latent Variables Models

Deep latent variable models combine the approximation abilities of deep ...
research
02/27/2017

Boosted Generative Models

We propose a new approach for using unsupervised boosting to create an e...
research
02/20/2019

Mixture Models for Diverse Machine Translation: Tricks of the Trade

Mixture models trained via EM are among the simplest, most widely used a...
research
02/13/2018

GILBO: One Metric to Measure Them All

We propose a simple, tractable lower bound on the mutual information con...
research
04/01/2020

SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models

Standard variational lower bounds used to train latent variable models p...

Please sign up or login with your details

Forgot password? Click here to reset