Furnishing Your Room by What You See: An End-to-End Furniture Set Retrieval Framework with Rich Annotated Benchmark Dataset

11/21/2019
by   Bingyuan Liu, et al.
9

Understanding interior scenes has attracted enormous interest in computer vision community. However, few works focus on the understanding of furniture within the scenes and a large-scale dataset is also lacked to advance the field. In this paper, we first fill the gap by presenting DeepFurniture, a richly annotated large indoor scene dataset, including 24k indoor images, 170k furniture instances and 20k unique furniture identities. On the dataset, we introduce a new benchmark, named furniture set retrieval. Given an indoor photo as input, the task requires to detect all the furniture instances and search a matched set of furniture identities. To address this challenging task, we propose a feature and context embedding based framework. It contains 3 major contributions: (1) An improved Mask-RCNN model with an additional mask-based classifier is introduced for better utilizing the mask information to relieve the occlusion problems in furniture detection context. (2) A multi-task style Siamese network is proposed to train the feature embedding model for retrieval, which is composed of a classification subnet supervised by self-clustered pseudo attributes and a verification subnet to estimate whether the input pair is matched. (3) In order to model the relationship of the furniture entities in an interior design, a context embedding model is employed to re-rank the retrieval results. Extensive experiments demonstrate the effectiveness of each module and the overall system.

READ FULL TEXT

page 2

page 5

page 7

page 11

page 12

research
09/25/2021

An embarrassingly simple comparison of machine learning algorithms for indoor scene classification

With the emergence of autonomous indoor robots, the computer vision task...
research
08/22/2023

ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes

We present ScanNet++, a large-scale dataset that couples together captur...
research
03/17/2022

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

Many basic indoor activities such as eating or writing are always conduc...
research
12/09/2019

Learning Structure-Appearance Joint Embedding for Indoor Scene Image Synthesis

Advanced image synthesis methods can generate photo-realistic images for...
research
01/12/2015

Tri-Subject Kinship Verification: Understanding the Core of A Family

One major challenge in computer vision is to go beyond the modeling of i...
research
10/18/2022

Nighttime Dehaze-Enhancement

In this paper, we introduce a new computer vision task called nighttime ...
research
11/29/2021

Instance-wise Occlusion and Depth Orders in Natural Scenes

In this paper, we introduce a new dataset, named InstaOrder, that can be...

Please sign up or login with your details

Forgot password? Click here to reset