Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects

10/27/2021
by   Jeffrey Ichnowski, et al.
5

The ability to grasp and manipulate transparent objects is a major challenge for robots. Existing depth cameras have difficulty detecting, localizing, and inferring the geometry of such objects. We propose using neural radiance fields (NeRF) to detect, localize, and infer the geometry of transparent objects with sufficient accuracy to find and grasp them securely. We leverage NeRF's view-independent learned density, place lights to increase specular reflections, and perform a transparency-aware depth-rendering that we feed into the Dex-Net grasp planner. We show how additional lights create specular reflections that improve the quality of the depth map, and test a setup for a robot workcell equipped with an array of cameras to perform transparent object manipulation. We also create synthetic and real datasets of transparent objects in real-world settings, including singulated objects, cluttered tables, and the top rack of a dishwasher. In each setting we show that NeRF and Dex-Net are able to reliably compute robust grasps on transparent objects, achieving 90 and 100 where baseline methods fail.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 5

page 6

page 7

page 8

page 9

page 10

09/10/2019

GlassLoc: Plenoptic Grasp Pose Detection in Transparent Clutter

Transparent objects are prevalent across many environments of interest f...
05/29/2020

Multi-modal Transfer Learning for Grasping Transparent and Specular Objects

State-of-the-art object grasping methods rely on depth sensing to plan r...
10/05/2019

Colored Transparent Object Matting from a Single Image Using Deep Learning

This paper proposes a deep learning based method for colored transparent...
10/06/2019

ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

Transparent objects are a common part of everyday life, yet they possess...
03/13/2018

TOM-Net: Learning Transparent Object Matting from a Single Image

This paper addresses the problem of transparent object matting. Existing...
03/29/2021

Refractive Light-Field Features for Curved Transparent Objects in Structure from Motion

Curved refractive objects are common in the human environment, and have ...
08/20/2021

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

Transparent objects, such as glass walls and doors, constitute architect...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.