Maximal Causes for Exponential Family Observables

03/04/2020
by   S. Hamid Mousavi, et al.
0

The data model of standard sparse coding assumes a weighted linear summation of latents to determine the mean of Gaussian observation noise. However, such a linear summation of latents is often at odds with non-Gaussian observables (e.g., means of the Bernoulli distribution have to lie in the unit interval), and also in the Gaussian case it can be difficult to justify for many types of data. Alternative superposition models (i.e., links between latents and observables) have therefore been investigated repeatedly. Here we show that using the maximum instead of a linear sum to link latents to observables allows for the derivation of very general and concise parameter update equations. Concretely, we derive a set of update equations that has the same functional form for all distributions of the exponential family (given that derivatives w.r.t. their parameters can be taken). Our results consequently allow for the development of latent variable models for commonly as well as for unusually distributed data. We numerically verify our analytical result assuming standard Gaussian, Gamma, Poisson, Bernoulli and Exponential distributions and point to some potential applications.

READ FULL TEXT

page 10

page 13

page 30

research
07/30/2018

Cumulative distribution functions for the five simplest natural exponential families

Suppose that the distribution of X_a belongs to a natural exponential fa...
research
06/19/2020

Latent variable modeling with random features

Gaussian process-based latent variable models are flexible and theoretic...
research
10/29/2018

Location and scale behaviour of the quantiles of a natural exponential family

Let P_0 be a probability on the real line generating a natural exponenti...
research
05/25/2017

Expectation Propagation for t-Exponential Family Using Q-Algebra

Exponential family distributions are highly useful in machine learning s...
research
05/26/2020

Truncating the Exponential with a Uniform Distribution

For a sample of Exponentially distributed durations we aim at point esti...
research
10/29/2019

Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures

Stein's method (Stein, 1973; 1981) is a powerful tool for statistical ap...
research
02/10/2018

Deep learning with t-exponential Bayesian kitchen sinks

Bayesian learning has been recently considered as an effective means of ...

Please sign up or login with your details

Forgot password? Click here to reset