Impact of Community Structure on Consensus Machine Learning

11/02/2020
by   Bao Huynh, et al.
0

Consensus dynamics support decentralized machine learning for data that is distributed across a cloud compute cluster or across the internet of things. In these and other settings, one seeks to minimize the time τ_ϵ required to obtain consensus within some ϵ>0 margin of error. τ_ϵ typically depends on the topology of the underlying communication network, and for many algorithms τ_ϵ depends on the second-smallest eigenvalue λ_2∈[0,1] of the network's normalized Laplacian matrix: τ_ϵ∼𝒪(λ_2^-1). Here, we analyze the effect on τ_ϵ of network community structure, which can arise when compute nodes/sensors are spatially clustered, for example. We study consensus machine learning over networks drawn from stochastic block models, which yield random networks that can contain heterogeneous communities with different sizes and densities. Using random matrix theory, we analyze the effects of communities on λ_2 and consensus, finding that λ_2 generally increases (i.e., τ_ϵ decreases) as one decreases the extent of community structure. We further observe that there exists a critical level of community structure at which τ_ϵ reaches a lower bound and is no longer limited by the presence of communities. We support our findings with empirical experiments for decentralized support vector machines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/23/2019

Decentralized Stochastic First-Order Methods for Large-scale Machine Learning

Decentralized consensus-based optimization is a general computational fr...
research
05/20/2016

Local communities obstruct global consensus: Naming game on multi-local-world networks

Community structure is essential for social communications, where indivi...
research
12/21/2018

Bootstrap percolation on the stochastic block model

We analyze the bootstrap percolation process on the stochastic block mod...
research
04/03/2019

Stochastic Blockmodels with Edge Information

Stochastic blockmodels allow us to represent networks in terms of a late...
research
10/06/2009

BRAINSTORMING: Consensus Learning in Practice

We present here an introduction to Brainstorming approach, that was rece...
research
12/21/2021

An Extension of InfoMap to Absorbing Random Walks

InfoMap is a popular approach for detecting densely connected "communiti...
research
06/07/2018

Stochastic Block Models are a Discrete Surface Tension

Networks, which represent agents and interactions between them, arise in...

Please sign up or login with your details

Forgot password? Click here to reset