Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs

08/19/2021
by   Helisa Dhamo, et al.
0

Controllable scene synthesis consists of generating 3D information that satisfy underlying specifications. Thereby, these specifications should be abstract, i.e. allowing easy user interaction, whilst providing enough interface for detailed control. Scene graphs are representations of a scene, composed of objects (nodes) and inter-object relationships (edges), proven to be particularly suited for this task, as they allow for semantic control on the generated content. Previous works tackling this task often rely on synthetic data, and retrieve object meshes, which naturally limits the generation capabilities. To circumvent this issue, we instead propose the first work that directly generates shapes from a scene graph in an end-to-end manner. In addition, we show that the same model supports scene modification, using the respective scene graph as interface. Leveraging Graph Convolutional Networks (GCN) we train a variational Auto-Encoder on top of the object and edge categories, as well as 3D shapes and scene layouts, allowing latter sampling of new scenes and shapes.

READ FULL TEXT

page 1

page 7

page 14

page 15

research
05/25/2023

CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graphs

Controllable scene synthesis aims to create interactive environments for...
research
08/12/2021

Unconditional Scene Graph Generation

Despite recent advancements in single-domain or single-object image gene...
research
08/08/2023

3D Scene Diffusion Guidance using Scene Graphs

Guided synthesis of high-quality 3D scenes is a challenging task. Diffus...
research
07/01/2022

Transforming Image Generation from Scene Graphs

Generating images from semantic visual knowledge is a challenging task, ...
research
11/10/2022

DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation

Graph representation of objects and their relations in a scene, known as...
research
11/30/2021

Hole-robust Wireframe Detection

"Wireframe" is a line segment based representation designed to well capt...
research
10/06/2016

Searching Scenes by Abstracting Things

In this paper we propose to represent a scene as an abstraction of 'thin...

Please sign up or login with your details

Forgot password? Click here to reset