DeepAI AI Chat
Log In Sign Up

Variational Inference and Sparsity in High-Dimensional Deep Gaussian Mixture Models

05/04/2021
by   Lucas Kock, et al.
0

Gaussian mixture models are a popular tool for model-based clustering, and mixtures of factor analyzers are Gaussian mixture models having parsimonious factor covariance structure for mixture components. There are several recent extensions of mixture of factor analyzers to deep mixtures, where the Gaussian model for the latent factors is replaced by a mixture of factor analyzers. This construction can be iterated to obtain a model with many layers. These deep models are challenging to fit, and we consider Bayesian inference using sparsity priors to further regularize the estimation. A scalable natural gradient variational inference algorithm is developed for fitting the model, and we suggest computationally efficient approaches to the architecture choice using overfitted mixtures where unnecessary components drop out in the estimation. In a number of simulated and two real examples, we demonstrate the versatility of our approach for high-dimensional problems, and demonstrate that the use of sparsity inducing priors can be helpful for obtaining improved clustering results.

READ FULL TEXT
11/18/2017

Deep Gaussian Mixture Models

Deep learning is a hierarchical inference method formed by subsequent mu...
01/14/2015

Dirichlet Process Parsimonious Mixtures for clustering

The parsimonious Gaussian mixture models, which exploit an eigenvalue de...
06/27/2012

Large Scale Variational Bayesian Inference for Structured Scale Mixture Models

Natural image statistics exhibit hierarchical dependencies across multip...
02/18/2022

Gaussian Mixture Convolution Networks

This paper proposes a novel method for deep learning based on the analyt...
11/25/2021

Path Guiding Using Spatio-Directional Mixture Models

We propose a learning-based method for light-path construction in path t...
06/22/2020

Deep Residual Mixture Models

We propose Deep Residual Mixture Models (DRMMs) which share the many des...
05/14/2018

Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures

Mixture models are widely used in Bayesian statistics and machine learni...