Instance-Level Task Parameters: A Robust Multi-task Weighting Framework

Recent works have shown that deep neural networks benefit from multi-task learning by learning a shared representation across several related tasks. However, performance of such systems depend on relative weighting between various losses involved during training. Prior works on loss weighting schemes assume that instances are equally easy or hard for all tasks. In order to break this assumption, we let the training process dictate the optimal weighting of tasks for every instance in the dataset. More specifically, we equip every instance in the dataset with a set of learnable parameters (instance-level task parameters) where the cardinality is equal to the number of tasks learned by the model. These parameters model the weighting of each task for an instance. They are updated by gradient descent and do not require hand-crafted rules. We conduct extensive experiments on SURREAL and CityScapes datasets, for human shape and pose estimation, depth estimation and semantic segmentation tasks. In these tasks, our approach outperforms recent dynamic loss weighting approaches, e.g. reducing surface estimation errors by 8.97 datasets where one or more tasks can have noisy annotations, the proposed method learns to prioritize learning from clean labels for a given task, e.g. reducing surface estimation errors by up to 60 reliably detect corrupt labels for a given task as a by-product from learned instance-level task parameters.


page 6

page 7


Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation

While many individual tasks in the domain of human analysis have recentl...

Multi-Loss Weighting with Coefficient of Variations

Many interesting tasks in machine learning and computer vision are learn...

UV R-CNN: Stable and Efficient Dense Human Pose Estimation

Dense pose estimation is a dense 3D prediction task for instance-level h...

Which Samples Should be Learned First: Easy or Hard?

An effective weighting scheme for training samples is essential for lear...

Object-Centric Multi-Task Learning for Human Instances

Human is one of the most essential classes in visual recognition tasks s...

AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving

Decision making in automated driving is highly specific to the environme...

Dynamic Task Weighting Methods for Multi-task Networks in Autonomous Driving Systems

Deep multi-task networks are of particular interest for autonomous drivi...

Please sign up or login with your details

Forgot password? Click here to reset