Visual Foresight Tree for Object Retrieval from Clutter with Nonprehensile Rearrangement

05/06/2021
by   Baichuan Huang, et al.
0

This paper considers the problem of retrieving an object from a set of tightly packed objects by using a combination of robotic pushing and grasping actions. Object retrieval from confined spaces that contain clutter is an important skill for robots in order to effectively operate in households and everyday environments. The proposed solution, Visual Foresight Tree (VFT), cleverly rearranges the clutter surrounding the target object so that it can be grasped easily. Rearrangement with nested non-prehensile actions is challenging as it requires predicting complex object interactions in a combinatorially large configuration space of multiple objects. We first show that a deep neural network can be trained to accurately predict the poses of the packed objects when the robot pushes one of them. The predictive network provides visual foresight and is used in a tree search as a state transition function in the space of scene images. The tree search returns a sequence of consecutive push actions that result in the best arrangement of the clutter for grasping the target object. Experiments in simulation and using a real robot and objects show that the proposed approach outperforms model-free techniques as well as model-based myopic methods both in terms of success rates and the number of executed actions, on several challenging tasks. A video introducing VFT, with robot experiments, is accessible at https://youtu.be/7cL-hmgvyec. Full source code will also be made available upon publication of this manuscript.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

research
05/17/2019

Object Rearrangement with Nested Nonprehensile Manipulation Actions

This paper considers the problem of rearrangement planning, i.e finding ...
research
09/11/2019

A Deep Learning Approach to Grasping the Invisible

We introduce a new problem named "grasping the invisible", where a robot...
research
06/24/2023

Push-MOG: Efficient Pushing to Consolidate Polygonal Objects for Multi-Object Grasping

Recently, robots have seen rapidly increasing use in homes and warehouse...
research
10/04/2022

Persistent Homology Guided Monte-Carlo Tree Search for Effective Non-Prehensile Manipulation

Performing object retrieval tasks in messy real-world workspaces involve...
research
06/25/2023

Effectively Rearranging Heterogeneous Objects on Cluttered Tabletops

Effectively rearranging heterogeneous objects constitutes a high-utility...
research
05/11/2020

Learning to Slide Unknown Objects with Differentiable Physics Simulations

We propose a new technique for pushing an unknown object from an initial...

Please sign up or login with your details

Forgot password? Click here to reset