When random initializations help: a study of variational inference for community detection

05/16/2019
by   Purnamrita Sarkar, et al.
0

Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM) with equal class sizes. Using batch co-ordinate ascent (BCAVI) for updates, we show different convergence behavior with respect to different initializations. When the parameters are known or estimated within a reasonable range and held fixed, we characterize conditions under which an initialization can converge to the ground truth. On the other hand, when the parameters need to be estimated iteratively, a random initialization will converge to an uninformative local optimum.

READ FULL TEXT

page 7

page 11

page 12

research
07/29/2019

A Theoretical Case Study of Structured Variational Inference for Community Detection

Mean-field variational inference (MFVI) has been widely applied in large...
research
10/30/2017

Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection

The mean field variational Bayes method is becoming increasingly popular...
research
01/12/2023

Variational Inference: Posterior Threshold Improves Network Clustering Accuracy in Sparse Regimes

Variational inference has been widely used in machine learning literatur...
research
07/09/2019

Scaling Limit of Neural Networks with the Xavier Initialization and Convergence to a Global Minimum

We analyze single-layer neural networks with the Xavier initialization i...
research
10/19/2012

A Generalized Mean Field Algorithm for Variational Inference in Exponential Families

The mean field methods, which entail approximating intractable probabili...
research
12/24/2020

On Batch Normalisation for Approximate Bayesian Inference

We study batch normalisation in the context of variational inference met...
research
05/26/2019

Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection

Landing probabilities (LP) of random walks (RW) over graphs encode rich ...

Please sign up or login with your details

Forgot password? Click here to reset