Visualizing and Understanding Sum-Product Networks

08/29/2016
by   Antonio Vergari, et al.
0

Sum-Product Networks (SPNs) are recently introduced deep tractable probabilistic models by which several kinds of inference queries can be answered exactly and in a tractable time. Up to now, they have been largely used as black box density estimators, assessed only by comparing their likelihood scores only. In this paper we explore and exploit the inner representations learned by SPNs. We do this with a threefold aim: first we want to get a better understanding of the inner workings of SPNs; secondly, we seek additional ways to evaluate one SPN model and compare it against other probabilistic models, providing diagnostic tools to practitioners; lastly, we want to empirically evaluate how good and meaningful the extracted representations are, as in a classic Representation Learning framework. In order to do so we revise their interpretation as deep neural networks and we propose to exploit several visualization techniques on their node activations and network outputs under different types of inference queries. To investigate these models as feature extractors, we plug some SPNs, learned in a greedy unsupervised fashion on image datasets, in supervised classification learning tasks. We extract several embedding types from node activations by filtering nodes by their type, by their associated feature abstraction level and by their scope. In a thorough empirical comparison we prove them to be competitive against those generated from popular feature extractors as Restricted Boltzmann Machines. Finally, we investigate embeddings generated from random probabilistic marginal queries as means to compare other tractable probabilistic models on a common ground, extending our experiments to Mixtures of Trees.

READ FULL TEXT

page 17

page 19

page 21

page 22

page 23

research
08/08/2016

Towards Representation Learning with Tractable Probabilistic Models

Probabilistic models learned as density estimators can be exploited in r...
research
07/14/2018

Tractable Querying and Learning in Hybrid Domains via Sum-Product Networks

Probabilistic representations, such as Bayesian and Markov networks, are...
research
05/17/2023

Tractable Probabilistic Graph Representation Learning with Graph-Induced Sum-Product Networks

We introduce Graph-Induced Sum-Product Networks (GSPNs), a new probabili...
research
04/02/2020

Sum-product networks: A survey

A sum-product network (SPN) is a probabilistic model, based on a rooted ...
research
01/31/2019

ProBO: a Framework for Using Probabilistic Programming in Bayesian Optimization

Optimizing an expensive-to-query function is a common task in science an...
research
08/08/2019

Random Sum-Product Forests with Residual Links

Tractable yet expressive density estimators are a key building block of ...
research
10/09/2017

Sum-Product Networks for Hybrid Domains

While all kinds of mixed data -from personal data, over panel and scient...

Please sign up or login with your details

Forgot password? Click here to reset