Efficient Variance-Reduced Learning for Fully Decentralized On-Device Intelligence

08/04/2017
by   Kun Yuan, et al.
0

This work develops a fully decentralized variance-reduced learning algorithm for on-device intelligence where nodes store and process the data locally and are only allowed to communicate with their immediate neighbors. In the proposed algorithm, there is no need for a central or master unit while the objective is to enable the dispersed nodes to learn the exact global model despite their limited localized interactions. The resulting algorithm is shown to have low memory requirement, guaranteed linear convergence, robustness to failure of links or nodes, scalability to the network size, and privacy-preserving properties. Moreover, the decentralized nature of the solution makes large-scale machine learning problems more tractable and also scalable since data is stored and processed locally at the nodes.

READ FULL TEXT

page 7

page 8

research
08/13/2018

COLA: Communication-Efficient Decentralized Linear Learning

Decentralized machine learning is a promising emerging paradigm in view ...
research
02/27/2023

MoDeST: Bridging the Gap between Federated and Decentralized Learning with Decentralized Sampling

Federated and decentralized machine learning leverage end-user devices f...
research
02/26/2019

On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communication

In parallel and distributed machine learning multiple nodes or processor...
research
02/23/2020

Quantized Push-sum for Gossip and Decentralized Optimization over Directed Graphs

We consider a decentralized stochastic learning problem where data point...
research
09/12/2020

Communication-efficient Decentralized Machine Learning over Heterogeneous Networks

In the last few years, distributed machine learning has been usually exe...
research
06/25/2020

Dual-Free Stochastic Decentralized Optimization with Variance Reduction

We consider the problem of training machine learning models on distribut...
research
07/29/2023

The effect of network topologies on fully decentralized learning: a preliminary investigation

In a decentralized machine learning system, data is typically partitione...

Please sign up or login with your details

Forgot password? Click here to reset