Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks

08/29/2019
by   Dae Ha Kim, et al.
0

One of the ways to improve the performance of a target task is to learn the transfer of abundant knowledge of a pre-trained network. However, learning of the pre-trained network requires high computation capability and large-scale labeled dataset. To mitigate the burden of large-scale labeling, learning in un/self-supervised manner can be a solution. In addition, using unsupervised multi-task learning, a generalized feature representation can be learned. However, unsupervised multi-task learning can be biased to a specific task. To overcome this problem, we propose the metric-based regularization term and temporal task ensemble (TTE) for multi-task learning. Since these two techniques prevent the entire network from learning in a state deviated to a specific task, it is possible to learn a generalized feature representation that appropriately reflects the characteristics of each task without biasing. Experimental results for three target tasks such as classification, object detection and embedding clustering prove that the TTE-based multi-task framework is more effective than the state-of-the-art (SOTA) method in improving the performance of a target task.

READ FULL TEXT
research
04/23/2022

Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning

Recent work has found that multi-task training with a large number of di...
research
12/02/2021

TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework using Self-Supervised Multi-Task Learning

In this paper, we propose TransMEF, a transformer-based multi-exposure i...
research
03/27/2022

Reinforcement Guided Multi-Task Learning Framework for Low-Resource Stereotype Detection

As large Pre-trained Language Models (PLMs) trained on large amounts of ...
research
07/20/2023

Pre-train, Adapt and Detect: Multi-Task Adapter Tuning for Camouflaged Object Detection

Camouflaged object detection (COD), aiming to segment camouflaged object...
research
11/24/2017

Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery

In human learning, it is common to use multiple sources of information j...
research
02/26/2020

Evolving Losses for Unsupervised Video Representation Learning

We present a new method to learn video representations from large-scale ...
research
04/02/2022

MRI-based Multi-task Decoupling Learning for Alzheimer's Disease Detection and MMSE Score Prediction: A Multi-site Validation

Accurately detecting Alzheimer's disease (AD) and predicting mini-mental...

Please sign up or login with your details

Forgot password? Click here to reset