DeepAI
Log In Sign Up

High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the looseness of the relaxation. To remedy this problem, we view jointly sparse multi-task learning as a specialized random effects model, and derive a convex relaxation approach that involves two steps. The first step learns the covariance matrix of the coefficients using a convex formulation which we refer to as sparse covariance coding; the second step solves a ridge regression problem with a sparse quadratic regularizer based on the covariance matrix obtained in the first step. It is shown that this approach produces an asymptotically optimal quadratic regularizer in the multitask learning setting when the number of tasks approaches infinity. Experimental results demonstrate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formulations

READ FULL TEXT

page 1

page 2

page 3

page 4

12/03/2019

Decentralised Sparse Multi-Task Regression

We consider a sparse multi-task regression framework for fitting a colle...
05/20/2018

Wasserstein regularization for sparse multi-task regression

Two important elements have driven recent innovation in the field of reg...
03/25/2013

On Sparsity Inducing Regularization Methods for Machine Learning

During the past years there has been an explosion of interest in learnin...
10/02/2015

Distributed Multitask Learning

We consider the problem of distributed multi-task learning, where each m...
03/04/2021

Multi-task Learning with High-Dimensional Noisy Images

Recent medical imaging studies have given rise to distinct but inter-rel...
07/06/2018

Multi-Task Learning with Incomplete Data for Healthcare

Multi-task learning is a type of transfer learning that trains multiple ...
10/18/2021

A Bayesian approach to multi-task learning with network lasso

Network lasso is a method for solving a multi-task learning problem thro...