Decentralized Multi-Task Learning Based on Extreme Learning Machines

04/25/2019
by   Yu Ye, et al.
0

In multi-task learning (MTL), related tasks learn jointly to improve generalization performance. To exploit the high learning speed of extreme learning machines (ELMs), we apply the ELM framework to the MTL problem, where the output weights of ELMs for all the tasks are learned collaboratively. We first present the ELM based MTL problem in the centralized setting, which is solved by the proposed MTL-ELM algorithm. Due to the fact that many data sets of different tasks are geo-distributed, decentralized machine learning is studied. We formulate the decentralized MTL problem based on ELM as majorized multi-block optimization with coupled bi-convex objective functions. To solve the problem, we propose the DMTL-ELM algorithm, which is a hybrid Jacobian and Gauss-Seidel Proximal multi-block alternating direction method of multipliers (ADMM). Further, to reduce the computation load of DMTL-ELM, DMTL-ELM with first-order approximation (FO-DMTL-ELM) is presented. Theoretical analysis shows that the convergence to the stationary point of DMTL-ELM and FO-DMTL-ELM can be guaranteed conditionally. Through simulations, we demonstrate the convergence of proposed MTL-ELM, DMTL-ELM, and FO-DMTL-ELM algorithms, and also show that they can outperform existing MTL methods. Moreover, by adjusting the dimension of hidden feature space, there exists a trade-off between communication load and learning accuracy for DMTL-ELM.

READ FULL TEXT
research
02/13/2018

Variable Selection and Task Grouping for Multi-Task Learning

We consider multi-task learning, which simultaneously learns related pre...
research
04/04/2012

Distributed Robust Power System State Estimation

Deregulation of energy markets, penetration of renewables, advanced mete...
research
01/24/2019

Communication-Efficient and Decentralized Multi-Task Boosting while Learning the Collaboration Graph

We study the decentralized machine learning scenario where many users co...
research
02/11/2018

Distributed Stochastic Multi-Task Learning with Graph Regularization

We propose methods for distributed graph-based multi-task learning that ...
research
02/14/2017

Efficient Multi-task Feature and Relationship Learning

In this paper we propose a multi-convex framework for multi-task learnin...
research
08/22/2019

Mobility-aware Content Preference Learning in Decentralized Caching Networks

Due to the drastic increase of mobile traffic, wireless caching is propo...
research
10/05/2020

The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings

This paper frames a general prediction system as an observer traveling a...

Please sign up or login with your details

Forgot password? Click here to reset