Learning noisy-OR Bayesian Networks with Max-Product Belief Propagation

01/31/2023
by   Antoine Dedieu, et al.
0

Noisy-OR Bayesian Networks (BNs) are a family of probabilistic graphical models which express rich statistical dependencies in binary data. Variational inference (VI) has been the main method proposed to learn noisy-OR BNs with complex latent structures (Jaakkola Jordan, 1999; Ji et al., 2020; Buhai et al., 2020). However, the proposed VI approaches either (a) use a recognition network with standard amortized inference that cannot induce “explaining-away”; or (b) assume a simple mean-field (MF) posterior which is vulnerable to bad local optima. Existing MF VI methods also update the MF parameters sequentially which makes them inherently slow. In this paper, we propose parallel max-product as an alternative algorithm for learning noisy-OR BNs with complex latent structures and we derive a fast stochastic training scheme that scales to large datasets. We evaluate both approaches on several benchmarks where VI is the state-of-the-art and show that our method (a) achieves better test performance than Ji et al. (2020) for learning noisy-OR BNs with hierarchical latent structures on large sparse real datasets; (b) recovers a higher number of ground truth parameters than Buhai et al. (2020) from cluttered synthetic scenes; and (c) solves the 2D blind deconvolution problem from Lazaro-Gredilla et al. (2021) and variant - including binary matrix factorization - while VI catastrophically fails and is up to two orders of magnitude slower.

READ FULL TEXT

page 1

page 12

page 25

page 27

research
01/23/2013

Multiplicative Factorization of Noisy-Max

The noisy-or and its generalization noisy-max have been utilized to redu...
research
07/04/2014

Identifying Higher-order Combinations of Binary Features

Finding statistically significant interactions between binary variables ...
research
12/14/2021

Branching Time Active Inference with Bayesian Filtering

Branching Time Active Inference (Champion et al., 2021b,a) is a framewor...
research
11/04/2021

Amortized Variational Inference for Simple Hierarchical Models

It is difficult to use subsampling with variational inference in hierarc...
research
05/24/2017

Proximity Variational Inference

Variational inference is a powerful approach for approximate posterior i...
research
07/03/2015

D-MFVI: Distributed Mean Field Variational Inference using Bregman ADMM

Bayesian models provide a framework for probabilistic modelling of compl...
research
06/29/2019

Approximate Inference in Structured Instances with Noisy Categorical Observations

We study the problem of recovering the latent ground truth labeling of a...

Please sign up or login with your details

Forgot password? Click here to reset