ManipulaTHOR: A Framework for Visual Object Manipulation

04/22/2021
by   Lucas Taylor, et al.
10

The domain of Embodied AI has recently witnessed substantial progress, particularly in navigating agents within their environments. These early successes have laid the building blocks for the community to tackle tasks that require agents to actively interact with objects in their environment. Object manipulation is an established research domain within the robotics community and poses several challenges including manipulator motion, grasping and long-horizon planning, particularly when dealing with oft-overlooked practical setups involving visually rich and complex scenes, manipulation using mobile agents (as opposed to tabletop manipulation), and generalization to unseen environments and objects. We propose a framework for object manipulation built upon the physics-enabled, visually rich AI2-THOR framework and present a new challenge to the Embodied AI community known as ArmPointNav. This task extends the popular point navigation task to object manipulation and offers new challenges including 3D obstacle avoidance, manipulating objects in the presence of occlusion, and multi-object manipulation that necessitates long term planning. Popular learning paradigms that are successful on PointNav challenges show promise, but leave a large room for improvement.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 12

research
11/16/2020

A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects

We present a framework for solving long-horizon planning problems involv...
research
03/15/2022

Object Manipulation via Visual Target Localization

Object manipulation is a critical skill required for Embodied AI agents ...
research
04/02/2020

Go Fetch: Mobile Manipulation in Unstructured Environments

With humankind facing new and increasingly large-scale challenges in the...
research
03/30/2021

Visual Room Rearrangement

There has been a significant recent progress in the field of Embodied AI...
research
01/23/2018

CHALET: Cornell House Agent Learning Environment

We present CHALET, a 3D house simulator with support for navigation and ...
research
09/14/2023

Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions

Perceiving and manipulating 3D articulated objects in diverse environmen...
research
10/06/2022

Embodied Referring Expression for Manipulation Question Answering in Interactive Environment

Embodied agents are expected to perform more complicated tasks in an int...

Please sign up or login with your details

Forgot password? Click here to reset