Robust Online Multi-Task Learning with Correlative and Personalized Structures

06/06/2017
by   Peng Yang, et al.
0

Multi-Task Learning (MTL) can enhance a classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline or batch setting, and suffers from expensive training cost and poor scalability. To address such inefficiency issues, online learning techniques have been applied to solve MTL problems. However, most existing algorithms of online MTL constrain task relatedness into a presumed structure via a single weight matrix, which is a strict restriction that does not always hold in practice. In this paper, we propose a robust online MTL framework that overcomes this restriction by decomposing the weight matrix into two components: the first one captures the low-rank common structure among tasks via a nuclear norm and the second one identifies the personalized patterns of outlier tasks via a group lasso. Theoretical analysis shows the proposed algorithm can achieve a sub-linear regret with respect to the best linear model in hindsight. Even though the above framework achieves good performance, the nuclear norm that simply adds all nonzero singular values together may not be a good low-rank approximation. To improve the results, we use a log-determinant function as a non-convex rank approximation. The gradient scheme is applied to optimize log-determinant function and can obtain a closed-form solution for this refined problem. Experimental results on a number of real-world applications verify the efficacy of our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/09/2021

New Tight Relaxations of Rank Minimization for Multi-Task Learning

Multi-task learning has been observed by many researchers, which suppose...
research
07/03/2015

LogDet Rank Minimization with Application to Subspace Clustering

Low-rank matrix is desired in many machine learning and computer vision ...
research
07/03/2017

Robust Cost-Sensitive Learning for Recommendation with Implicit Feedback

Recommendation is the task of improving customer experience through pers...
research
02/12/2020

Deep Multi-Task Learning via Generalized Tensor Trace Norm

The trace norm is widely used in multi-task learning as it can discover ...
research
06/12/2015

Robust Structured Low-Rank Approximation on the Grassmannian

Over the past years Robust PCA has been established as a standard tool f...
research
07/25/2017

A Survey on Multi-Task Learning

Multi-Task Learning (MTL) is a learning paradigm in machine learning and...
research
03/27/2020

Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations

The learning of the deep networks largely relies on the data with human-...

Please sign up or login with your details

Forgot password? Click here to reset