Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity

09/28/2020
by   William Agnew, et al.
2

Learning-based 3D object reconstruction enables single- or few-shot estimation of 3D object models. For robotics, this holds the potential to allow model-based methods to rapidly adapt to novel objects and scenes. Existing 3D reconstruction techniques optimize for visual reconstruction fidelity, typically measured by chamfer distance or voxel IOU. We find that when applied to realistic, cluttered robotics environments, these systems produce reconstructions with low physical realism, resulting in poor task performance when used for model-based control. We propose ARM, an amodal 3D reconstruction system that introduces (1) a stability prior over object shapes, (2) a connectivity prior, and (3) a multi-channel input representation that allows for reasoning over relationships between groups of objects. By using these priors over the physical properties of objects, our system improves reconstruction quality not just by standard visual metrics, but also performance of model-based control on a variety of robotics manipulation tasks in challenging, cluttered environments. Code is available at github.com/wagnew3/ARM.

READ FULL TEXT

page 2

page 3

page 4

page 7

research
07/05/2023

FOCUS: Object-Centric World Models for Robotics Manipulation

Understanding the world in terms of objects and the possible interplays ...
research
04/19/2023

Anything-3D: Towards Single-view Anything Reconstruction in the Wild

3D reconstruction from a single-RGB image in unconstrained real-world sc...
research
07/19/2021

Ab Initio Particle-based Object Manipulation

This paper presents Particle-based Object Manipulation (Prompt), a new a...
research
04/21/2022

Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research

Autonomous fabric manipulation is a longstanding challenge in robotics, ...
research
10/05/2021

3D-MOV: Audio-Visual LSTM Autoencoder for 3D Reconstruction of Multiple Objects from Video

3D object reconstructions of transparent and concave structured objects,...
research
02/01/2022

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

Accurate object rearrangement from vision is a crucial problem for a wid...
research
09/15/2016

Visual Stability Prediction and Its Application to Manipulation

Understanding physical phenomena is a key competence that enables humans...

Please sign up or login with your details

Forgot password? Click here to reset