Distributed Primal-Dual Optimization for Online Multi-Task Learning

04/02/2020
by   Peng Yang, et al.
0

Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness. To address these issues, in this paper we consider a setting where multiple tasks are geographically located in different places, where one task can synchronize data with others to leverage knowledge of related tasks. Specifically, we propose an adaptive primal-dual algorithm, which not only captures task-specific noise in adversarial learning but also carries out a projection-free update with runtime efficiency. Moreover, our model is well-suited to decentralized periodic-connected tasks as it allows the energy-starved or bandwidth-constraint tasks to postpone the update. Theoretical results demonstrate the convergence guarantee of our distributed algorithm with an optimal regret. Empirical results confirm that the proposed model is highly effective on various real-world datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/30/2017

Federated Multi-Task Learning

Federated learning poses new statistical and systems challenges in train...
research
10/27/2022

Multi-task Bias-Variance Trade-off Through Functional Constraints

Multi-task learning aims to acquire a set of functions, either regressor...
research
01/06/2023

Task Aware Feature Extraction Framework for Sequential Dependence Multi-Task Learning

Multi-task learning (MTL) has been successfully implemented in many real...
research
10/24/2020

Multi-task Supervised Learning via Cross-learning

In this paper we consider a problem known as multi-task learning, consis...
research
02/09/2017

Coordinated Online Learning With Applications to Learning User Preferences

We study an online multi-task learning setting, in which instances of re...
research
01/04/2023

Multi-Task Learning with Prior Information

Multi-task learning aims to boost the generalization performance of mult...
research
10/02/2015

Distributed Multitask Learning

We consider the problem of distributed multi-task learning, where each m...

Please sign up or login with your details

Forgot password? Click here to reset