UniNet: A Unified Scene Understanding Network and Exploring Multi-Task Relationships through the Lens of Adversarial Attacks

08/10/2021
by   NareshKumar Gurulingan, et al.
10

Scene understanding is crucial for autonomous systems which intend to operate in the real world. Single task vision networks extract information only based on some aspects of the scene. In multi-task learning (MTL), on the other hand, these single tasks are jointly learned, thereby providing an opportunity for tasks to share information and obtain a more comprehensive understanding. To this end, we develop UniNet, a unified scene understanding network that accurately and efficiently infers vital vision tasks including object detection, semantic segmentation, instance segmentation, monocular depth estimation, and monocular instance depth prediction. As these tasks look at different semantic and geometric information, they can either complement or conflict with each other. Therefore, understanding inter-task relationships can provide useful cues to enable complementary information sharing. We evaluate the task relationships in UniNet through the lens of adversarial attacks based on the notion that they can exploit learned biases and task interactions in the neural network. Extensive experiments on the Cityscapes dataset, using untargeted and targeted attacks reveal that semantic tasks strongly interact amongst themselves, and the same holds for geometric tasks. Additionally, we show that the relationship between semantic and geometric tasks is asymmetric and their interaction becomes weaker as we move towards higher-level representations.

READ FULL TEXT

page 6

page 8

page 13

page 14

page 15

page 16

page 17

research
01/19/2021

SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular images

Depth estimation and semantic segmentation play essential roles in scene...
research
08/25/2021

Multi-task learning from fixed-wing UAV images for 2D/3D city modeling

Single-task learning in artificial neural networks will be able to learn...
research
04/03/2023

Joint 2D-3D Multi-Task Learning on Cityscapes-3D: 3D Detection, Segmentation, and Depth Estimation

This report serves as a supplementary document for TaskPrompter, detaili...
research
07/14/2022

Adversarial Attacks on Monocular Pose Estimation

Advances in deep learning have resulted in steady progress in computer v...
research
08/06/2023

Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities

Safety and efficiency are paramount in healthcare facilities where the l...
research
06/02/2023

Towards In-context Scene Understanding

In-context learningx2013the ability to configure a model's behavior with...

Please sign up or login with your details

Forgot password? Click here to reset