Log In Sign Up

High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the looseness of the relaxation. To remedy this problem, we view jointly sparse multi-task learning as a specialized random effects model, and derive a convex relaxation approach that involves two steps. The first step learns the covariance matrix of the coefficients using a convex formulation which we refer to as sparse covariance coding; the second step solves a ridge regression problem with a sparse quadratic regularizer based on the covariance matrix obtained in the first step. It is shown that this approach produces an asymptotically optimal quadratic regularizer in the multitask learning setting when the number of tasks approaches infinity. Experimental results demonstrate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formulations


page 1

page 2

page 3

page 4


Decentralised Sparse Multi-Task Regression

We consider a sparse multi-task regression framework for fitting a colle...

Wasserstein regularization for sparse multi-task regression

Two important elements have driven recent innovation in the field of reg...

On Sparsity Inducing Regularization Methods for Machine Learning

During the past years there has been an explosion of interest in learnin...

Distributed Multitask Learning

We consider the problem of distributed multi-task learning, where each m...

Multi-task Learning with High-Dimensional Noisy Images

Recent medical imaging studies have given rise to distinct but inter-rel...

Multi-Task Learning with Incomplete Data for Healthcare

Multi-task learning is a type of transfer learning that trains multiple ...

A Bayesian approach to multi-task learning with network lasso

Network lasso is a method for solving a multi-task learning problem thro...