Log In Sign Up

Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

by   Alex Kendall, et al.

Numerous deep learning applications benefit from multi-task learning with multiple regression and classification objectives. In this paper we make the observation that the performance of such systems is strongly dependent on the relative weighting between each task's loss. Tuning these weights by hand is a difficult and expensive process, making multi-task learning prohibitive in practice. We propose a principled approach to multi-task deep learning which weighs multiple loss functions by considering the homoscedastic uncertainty of each task. This allows us to simultaneously learn various quantities with different units or scales in both classification and regression settings. We demonstrate our model learning per-pixel depth regression, semantic and instance segmentation from a monocular input image. Perhaps surprisingly, we show our model can learn multi-task weightings and outperform separate models trained individually on each task.


page 8

page 12

page 13


Multi-Task Learning in Histo-pathology for Widely Generalizable Model

In this work we show preliminary results of deep multi-task learning in ...

Multi-Task Variational Information Bottleneck

In this paper we propose a multi-task deep learning model called multi-t...

MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning

Multi-task learning is commonly used in autonomous driving for solving v...

HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxiliary Tasks

Multi-task learning (MTL) can improve performance on a task by sharing r...

Infinite-Task Learning with Vector-Valued RKHSs

Machine learning has witnessed the tremendous success of solving tasks d...

On Steering Multi-Annotations per Sample for Multi-Task Learning

The study of multi-task learning has drawn great attention from the comm...

Nowruz at SemEval-2022 Task 7: Tackling Cloze Tests with Transformers and Ordinal Regression

This paper outlines the system using which team Nowruz participated in S...

Code Repositories


A multi-task learning example for the paper

view repo