DeepAI AI Chat
Log In Sign Up

CompositeTasking: Understanding Images by Spatial Composition of Tasks

by   Nikola Popovic, et al.

We define the concept of CompositeTasking as the fusion of multiple, spatially distributed tasks, for various aspects of image understanding. Learning to perform spatially distributed tasks is motivated by the frequent availability of only sparse labels across tasks, and the desire for a compact multi-tasking network. To facilitate CompositeTasking, we introduce a novel task conditioning model – a single encoder-decoder network that performs multiple, spatially varying tasks at once. The proposed network takes a pair of an image and a set of pixel-wise dense tasks as inputs, and makes the task related predictions for each pixel, which includes the decision of applying which task where. As to the latter, we learn the composition of tasks that needs to be performed according to some CompositeTasking rules. It not only offers us a compact network for multi-tasking, but also allows for task-editing. The strength of the proposed method is demonstrated by only having to supply sparse supervision per task. The obtained results are on par with our baselines that use dense supervision and a multi-headed multi-tasking design. The source code will be made publicly available at .


page 7

page 8

page 12

page 13

page 14

page 15

page 16

page 18


Spatially Multi-conditional Image Generation

In most scenarios, conditional image generation can be thought of as an ...

Guided Collaborative Training for Pixel-wise Semi-Supervised Learning

We investigate the generalization of semi-supervised learning (SSL) to d...

Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform

We propose a versatile deep image compression network based on Spatial F...

Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matching

Dense prediction tasks are a fundamental class of problems in computer v...

Dense Transformer Networks

The key idea of current deep learning methods for dense prediction is to...

DPF: Learning Dense Prediction Fields with Weak Supervision

Nowadays, many visual scene understanding problems are addressed by dens...

Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton

In this paper, we solve three low-level pixel-wise vision problems, incl...