DeepAI AI Chat
Log In Sign Up

Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains

05/31/2019
by   Elliot Meyerson, et al.
The University of Texas at Austin
cognizant
0

As deep learning applications continue to become more diverse, an interesting question arises: Can general problem solving arise from jointly learning several such diverse tasks? To approach this question, deep multi-task learning is extended in this paper to the setting where there is no obvious overlap between task architectures. The idea is that any set of (architecture,task) pairs can be decomposed into a set of potentially related subproblems, whose sharing is optimized by an efficient stochastic algorithm. The approach is first validated in a classic synthetic multi-task learning benchmark, and then applied to sharing across disparate architectures for vision, NLP, and genomics tasks. It discovers regularities across these domains, encodes them into sharable modules, and combines these modules systematically to improve performance in the individual tasks. The results confirm that sharing learned functionality across diverse domains and architectures is indeed beneficial, thus establishing a key ingredient for general problem solving in the future.

READ FULL TEXT

page 6

page 8

06/27/2012

Learning Task Grouping and Overlap in Multi-task Learning

In the paradigm of multi-task learning, mul- tiple related prediction ta...
01/19/2020

Gradient Surgery for Multi-Task Learning

While deep learning and deep reinforcement learning (RL) systems have de...
05/23/2017

Sluice networks: Learning what to share between loosely related tasks

Multi-task learning is partly motivated by the observation that humans b...
09/27/2022

Design Perspectives of Multitask Deep Learning Models and Applications

In recent years, multi-task learning has turned out to be of great succe...
04/11/2020

Multi-task Learning via Adaptation to Similar Tasks for Mortality Prediction of Diverse Rare Diseases

Mortality prediction of diverse rare diseases using electronic health re...
05/18/2020

Efficient Image Gallery Representations at Scale Through Multi-Task Learning

Image galleries provide a rich source of diverse information about a pro...
04/29/2020

Emerging Relation Network and Task Embedding for Multi-Task Regression Problems

Multi-task learning (mtl) provides state-of-the-art results in many appl...