Using Depth for Improving Referring Expression Comprehension in Real-World Environments

07/09/2021
by   Fethiye Irmak Doğan, et al.
0

In a human-robot collaborative task where a robot helps its partner by finding described objects, the depth dimension plays a critical role in successful task completion. Existing studies have mostly focused on comprehending the object descriptions using RGB images. However, 3-dimensional space perception that includes depth information is fundamental in real-world environments. In this work, we propose a method to identify the described objects considering depth dimension data. Using depth features significantly improves performance in scenes where depth data is critical to disambiguate the objects and across our whole evaluation dataset that contains objects that can be specified with and without the depth dimension.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 6

page 7

09/30/2021

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects

The basis of many object manipulation algorithms is RGB-D input. Yet, co...
04/15/2019

Learning to Generate Unambiguous Spatial Referring Expressions for Real-World Environments

Referring to objects in a natural and unambiguous manner is crucial for ...
05/30/2019

Grounding Language Attributes to Objects using Bayesian Eigenobjects

We develop a system to disambiguate objects based on simple physical des...
04/02/2019

Improving Robot Success Detection using Static Object Data

We use static object data to improve success detection for stacking obje...
07/12/2021

Leveraging Explainability for Comprehending Referring Expressions in the Real World

For effective human-robot collaboration, it is crucial for robots to und...
05/02/2019

Improving Visual Relation Detection using Depth Maps

State of the art visual relation detection methods have been relying on ...
01/23/2017

Identification of Unmodeled Objects from Symbolic Descriptions

Successful human-robot cooperation hinges on each agent's ability to pro...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.