TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

by   Mutian Xu, et al.

Many basic indoor activities such as eating or writing are always conducted upon different tabletops (e.g., coffee tables, writing desks). It is indispensable to understanding tabletop scenes in 3D indoor scene parsing applications. Unfortunately, it is hard to meet this demand by directly deploying data-driven algorithms, since 3D tabletop scenes are rarely available in current datasets. To remedy this defect, we introduce TO-Scene, a large-scale dataset focusing on tabletop scenes, which contains 20,740 scenes with three variants. To acquire the data, we design an efficient and scalable framework, where a crowdsourcing UI is developed to transfer CAD objects onto tables from ScanNet, then the output tabletop scenes are simulated into real scans and annotated automatically. Further, a tabletop-aware learning strategy is proposed for better perceiving the small-sized tabletop instances. Notably, we also provide a real scanned test set TO-Real to verify the practical value of TO-Scene. Experiments show that the algorithms trained on TO-Scene indeed work on the realistic test data, and our proposed tabletop-aware learning strategy greatly improves the state-of-the-art results on both 3D semantic segmentation and object detection tasks. TO-Scene and TO-Real, plus Web UI, will all be publicly available.


page 1

page 5

page 16

page 17

page 19

page 22

page 24

page 28


Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks

Indoor scene understanding is central to applications such as robot navi...

Information Pursuit: A Bayesian Framework for Sequential Scene Parsing

Despite enormous progress in object detection and classification, the pr...

A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation

Recent advances of 3D acquisition devices have enabled large-scale acqui...

A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing

A key algorithm for understanding the world is material segmentation, wh...

Furnishing Your Room by What You See: An End-to-End Furniture Set Retrieval Framework with Rich Annotated Benchmark Dataset

Understanding interior scenes has attracted enormous interest in compute...

LUMINOUS: Indoor Scene Generation for Embodied AI Challenges

Learning-based methods for training embodied agents typically require a ...

SketchyScene: Richly-Annotated Scene Sketches

We contribute the first large-scale dataset of scene sketches, SketchySc...

Please sign up or login with your details

Forgot password? Click here to reset