On the Latent Variable Interpretation in Sum-Product Networks

01/22/2016
by   Robert Peharz, et al.
0

One of the central themes in Sum-Product networks (SPNs) is the interpretation of sum nodes as marginalized latent variables (LVs). This interpretation yields an increased syntactic or semantic structure, allows the application of the EM algorithm and to efficiently perform MPE inference. In literature, the LV interpretation was justified by explicitly introducing the indicator variables corresponding to the LVs' states. However, as pointed out in this paper, this approach is in conflict with the completeness condition in SPNs and does not fully specify the probabilistic model. We propose a remedy for this problem by modifying the original approach for introducing the LVs, which we call SPN augmentation. We discuss conditional independencies in augmented SPNs, formally establish the probabilistic interpretation of the sum-weights and give an interpretation of augmented SPNs as Bayesian networks. Based on these results, we find a sound derivation of the EM algorithm for SPNs. Furthermore, the Viterbi-style algorithm for MPE proposed in literature was never proven to be correct. We show that this is indeed a correct algorithm, when applied to selective SPNs, and in particular when applied to augmented SPNs. Our theoretical results are confirmed in experiments on synthetic data and 103 real-world datasets.

READ FULL TEXT
research
11/14/2022

Treatment-RSPN: Recurrent Sum-Product Networks for Sequential Treatment Regimes

Sum-product networks (SPNs) have recently emerged as a novel deep learni...
research
09/04/2017

Inverse Ising problem in continuous time: A latent variable approach

We consider the inverse Ising problem, i.e. the inference of network cou...
research
02/10/2018

Probabilistic Canonical Correlation Analysis: A Whitening Approach

Canonical correlation analysis (CCA) is a classic and widely used statis...
research
05/15/2022

Learning Shared Kernel Models: the Shared Kernel EM algorithm

Expectation maximisation (EM) is an unsupervised learning method for est...
research
10/29/2019

Scalable Inference for Nonparametric Hawkes Process Using Pólya-Gamma Augmentation

In this paper, we consider the sigmoid Gaussian Hawkes process model: th...
research
01/06/2015

On the Relationship between Sum-Product Networks and Bayesian Networks

In this paper, we establish some theoretical connections between Sum-Pro...
research
01/18/2019

Optimized Realization of Bayesian Networks in Reduced Normal Form using Latent Variable Model

Bayesian networks in their Factor Graph Reduced Normal Form (FGrn) are a...

Please sign up or login with your details

Forgot password? Click here to reset