Structure from Action: Learning Interactions for Articulated Object 3D Structure Discovery

07/19/2022
by   Neil Nie, et al.
0

Articulated objects are abundant in daily life. Discovering their parts, joints, and kinematics is crucial for robots to interact with these objects. We introduce Structure from Action (SfA), a framework that discovers the 3D part geometry and joint parameters of unseen articulated objects via a sequence of inferred interactions. Our key insight is that 3D interaction and perception should be considered in conjunction to construct 3D articulated CAD models, especially in the case of categories not seen during training. By selecting informative interactions, SfA discovers parts and reveals initially occluded surfaces, like the inside of a closed drawer. By aggregating visual observations in 3D, SfA accurately segments multiple parts, reconstructs part geometry, and infers all joint parameters in a canonical coordinate frame. Our experiments demonstrate that a single SfA model trained in simulation can generalize to many unseen object categories with unknown kinematic structures and to real-world objects. Code and data will be publicly available.

READ FULL TEXT

page 1

page 4

page 6

research
06/28/2021

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

Perceiving and manipulating 3D articulated objects (e.g., cabinets, door...
research
12/21/2021

Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects

Rendering articulated objects while controlling their poses is critical ...
research
05/03/2021

Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

People often use physical intuition when manipulating articulated object...
research
12/01/2020

Visual Identification of Articulated Object Parts

As autonomous robots interact and navigate around real-world environment...
research
09/14/2023

Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects

Articulated object manipulation is a fundamental yet challenging task in...
research
12/01/2021

AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions

Perceiving and interacting with 3D articulated objects, such as cabinets...
research
07/31/2023

Part-level Scene Reconstruction Affords Robot Interaction

Existing methods for reconstructing interactive scenes primarily focus o...

Please sign up or login with your details

Forgot password? Click here to reset