Robust Task Clustering for Deep Many-Task Learning

08/26/2017
by   Mo Yu, et al.
0

We investigate task clustering for deep-learning based multi-task and few-shot learning in a many-task setting. We propose a new method to measure task similarities with cross-task transfer performance matrix for the deep learning scenario. Although this matrix provides us critical information regarding similarity between tasks, its asymmetric property and unreliable performance scores can affect conventional clustering methods adversely. Additionally, the uncertain task-pairs, i.e., the ones with extremely asymmetric transfer scores, may collectively mislead clustering algorithms to output an inaccurate task-partition. To overcome these limitations, we propose a novel task-clustering algorithm by using the matrix completion technique. The proposed algorithm constructs a partially-observed similarity matrix based on the certainty of cluster membership of the task-pairs. We then use a matrix completion algorithm to complete the similarity matrix. Our theoretical analysis shows that under mild constraints, the proposed algorithm will perfectly recover the underlying "true" similarity matrix with a high probability. Our results show that the new task clustering method can discover task clusters for training flexible and superior neural network models in a multi-task learning setup for sentiment classification and dialog intent classification tasks. Our task clustering approach also extends metric-based few-shot learning methods to adapt multiple metrics, which demonstrates empirical advantages when the tasks are diverse.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/20/2023

Transductive Matrix Completion with Calibration for Multi-Task Learning

Multi-task learning has attracted much attention due to growing multi-pu...
research
09/02/2020

Clustering of Nonnegative Data and an Application to Matrix Completion

In this paper, we propose a simple algorithm to cluster nonnegative data...
research
07/04/2018

Regularizing Autoencoder-Based Matrix Completion Models via Manifold Learning

Autoencoders are popular among neural-network-based matrix completion mo...
research
06/23/2020

Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning

Although recent multi-task learning methods have shown to be effective i...
research
04/14/2015

Clustering Assisted Fundamental Matrix Estimation

In computer vision, the estimation of the fundamental matrix is a basic ...
research
05/25/2018

Randomized Robust Matrix Completion for the Community Detection Problem

This paper focuses on the unsupervised clustering of large partially obs...
research
04/19/2021

Automated problem setting selection in multi-target prediction with AutoMTP

Algorithm Selection (AS) is concerned with the selection of the best-sui...

Please sign up or login with your details

Forgot password? Click here to reset