Multi-task Supervised Learning via Cross-learning

10/24/2020
by   Juan Cervino, et al.
0

In this paper we consider a problem known as multi-task learning, consisting of fitting a set of classifier or regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across different domains help improving the learning performance at each other task. First, we present a simplified case in which the goal is to estimate the means of two Gaussian variables, for the purpose of gaining some insights on the advantage of the proposed cross-learning strategy. Then we provide a stochastic projected gradient algorithm to perform cross-learning over a generic loss function. If the number of parameters is large, then the projection step becomes computationally expensive. To avoid this situation, we derive a primal-dual algorithm that exploits the structure of the dual problem, achieving a formulation whose complexity only depends on the number of tasks. Preliminary numerical experiments for image classification by neural networks trained on a dataset divided in different domains corroborate that the cross-learned function outperforms both the task-specific and the consensus approaches.

READ FULL TEXT
research
10/27/2022

Multi-task Bias-Variance Trade-off Through Functional Constraints

Multi-task learning aims to acquire a set of functions, either regressor...
research
06/30/2015

Framework for Multi-task Multiple Kernel Learning and Applications in Genome Analysis

We present a general regularization-based framework for Multi-task learn...
research
04/02/2020

Distributed Primal-Dual Optimization for Online Multi-Task Learning

Conventional online multi-task learning algorithms suffer from two criti...
research
11/15/2022

Cross-Stitched Multi-task Dual Recursive Networks for Unified Single Image Deraining and Desnowing

We present the Cross-stitched Multi-task Unified Dual Recursive Network ...
research
09/15/2023

Projected Task-Specific Layers for Multi-Task Reinforcement Learning

Multi-task reinforcement learning could enable robots to scale across a ...
research
02/22/2021

Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer

We propose UniT, a Unified Transformer model to simultaneously learn the...
research
03/16/2020

Interpretable MTL from Heterogeneous Domains using Boosted Tree

Multi-task learning (MTL) aims at improving the generalization performan...

Please sign up or login with your details

Forgot password? Click here to reset