Task-specific Scene Structure Representations

01/02/2023
by   Jisu Shin, et al.
0

Understanding the informative structures of scenes is essential for low-level vision tasks. Unfortunately, it is difficult to obtain a concrete visual definition of the informative structures because influences of visual features are task-specific. In this paper, we propose a single general neural network architecture for extracting task-specific structure guidance for scenes. To do this, we first analyze traditional spectral clustering methods, which computes a set of eigenvectors to model a segmented graph forming small compact structures on image domains. We then unfold the traditional graph-partitioning problem into a learnable network, named Scene Structure Guidance Network (SSGNet), to represent the task-specific informative structures. The SSGNet yields a set of coefficients of eigenvectors that produces explicit feature representations of image structures. In addition, our SSGNet is light-weight (∼ 55K parameters), and can be used as a plug-and-play module for off-the-shelf architectures. We optimize the SSGNet without any supervision by proposing two novel training losses that enforce task-specific scene structure generation during training. Our main contribution is to show that such a simple network can achieve state-of-the-art results for several low-level vision applications including joint upsampling and image denoising. We also demonstrate that our SSGNet generalizes well on unseen datasets, compared to existing methods which use structural embedding frameworks. Our source codes are available at https://github.com/jsshin98/SSGNet.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

research
03/20/2023

Explicit Visual Prompting for Low-Level Structure Segmentations

We consider the generic problem of detecting low-level structures in ima...
research
12/27/2019

Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

In this paper, we address the task of semantic-guided scene generation. ...
research
04/20/2020

LSM: Learning Subspace Minimization for Low-level Vision

We study the energy minimization problem in low-level vision tasks from ...
research
05/29/2023

Explicit Visual Prompting for Universal Foreground Segmentations

Foreground segmentation is a fundamental problem in computer vision, whi...
research
01/09/2023

SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks

Understanding traffic scenes requires considering heterogeneous informat...
research
12/05/2018

Learning to Compose Dynamic Tree Structures for Visual Contexts

We propose to compose dynamic tree structures that place the objects in ...
research
06/02/2023

Bilevel Fast Scene Adaptation for Low-Light Image Enhancement

Enhancing images in low-light scenes is a challenging but widely concern...

Please sign up or login with your details

Forgot password? Click here to reset