Learning Sparse Sharing Architectures for Multiple Tasks

11/12/2019
by   Tianxiang Sun, et al.
0

Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing. How choosing a suitable sharing mechanism depends on the relations among the tasks, which is not easy since it is difficult to understand the underlying shared factors among these tasks. In this paper, we propose a novel parameter sharing mechanism, named Sparse Sharing. Given multiple tasks, our approach automatically finds a sparse sharing structure. We start with an over-parameterized base network, from which each task extracts a subnetwork. The subnetworks of multiple tasks are partially overlapped and trained in parallel. We show that both hard sharing and hierarchical sharing can be formulated as particular instances of the sparse sharing framework. We conduct extensive experiments on three sequence labeling tasks. Compared with single-task models and three typical multi-task learning baselines, our proposed approach achieves consistent improvement while requiring fewer parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/01/2020

Boosting share routing for multi-task learning

Multi-task learning (MTL) aims to make full use of the knowledge contain...
research
05/26/2023

DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning

Multi-task networks rely on effective parameter sharing to achieve robus...
research
04/05/2019

Learning Task Relatedness in Multi-Task Learning for Images in Context

Multimedia applications often require concurrent solutions to multiple t...
research
10/04/2017

Multitask Learning using Task Clustering with Applications to Predictive Modeling and GWAS of Plant Varieties

Inferring predictive maps between multiple input and multiple output var...
research
08/23/2023

OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes

In this work, we seek to predict camera poses across scenes with a multi...
research
08/22/2020

LT4REC:A Lottery Ticket Hypothesis Based Multi-task Practice for Video Recommendation System

Click-through rate prediction (CTR) and post-click conversion rate predi...
research
12/03/2014

Curriculum Learning of Multiple Tasks

Sharing information between multiple tasks enables algorithms to achieve...

Please sign up or login with your details

Forgot password? Click here to reset