Deep Elastic Networks with Model Selection for Multi-Task Learning

09/11/2019
by   Chanho Ahn, et al.
0

In this work, we consider the problem of instance-wise dynamic network model selection for multi-task learning. To this end, we propose an efficient approach to exploit a compact but accurate model in a backbone architecture for each instance of all tasks. The proposed method consists of an estimator and a selector. The estimator is based on a backbone architecture and structured hierarchically. It can produce multiple different network models of different configurations in a hierarchical structure. The selector chooses a model dynamically from a pool of candidate models given an input instance. The selector is a relatively small-size network consisting of a few layers, which estimates a probability distribution over the candidate models when an input instance of a task is given. Both estimator and selector are jointly trained in a unified learning framework in conjunction with a sampling-based learning strategy, without additional computation steps. We demonstrate the proposed approach for several image classification tasks compared to existing approaches performing model selection or learning multiple tasks. Experimental results show that our approach gives not only outstanding performance compared to other competitors but also the versatility to perform instance-wise model selection for multiple tasks.

READ FULL TEXT

page 8

page 13

research
05/09/2020

Multi-Task Learning in Histo-pathology for Widely Generalizable Model

In this work we show preliminary results of deep multi-task learning in ...
research
05/26/2023

DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning

Multi-task networks rely on effective parameter sharing to achieve robus...
research
03/10/2022

A Tree-Structured Multi-Task Model Recommender

Tree-structured multi-task architectures have been employed to jointly t...
research
02/20/2017

Learning to Multi-Task by Active Sampling

One of the long-standing challenges in Artificial Intelligence for learn...
research
12/16/2015

Blockout: Dynamic Model Selection for Hierarchical Deep Networks

Most deep architectures for image classification--even those that are tr...
research
04/09/2019

Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks

Deep networks consume a large amount of memory by their nature. A natura...
research
06/04/2018

Agreement-based Learning

Model selection is a problem that has occupied machine learning research...

Please sign up or login with your details

Forgot password? Click here to reset