3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding

03/30/2021
by   Shengheng Deng, et al.
7

The ability to understand the ways to interact with objects from visual cues, a.k.a. visual affordance, is essential to vision-guided robotic research. This involves categorizing, segmenting and reasoning of visual affordance. Relevant studies in 2D and 2.5D image domains have been made previously, however, a truly functional understanding of object affordance requires learning and prediction in the 3D physical domain, which is still absent in the community. In this work, we present a 3D AffordanceNet dataset, a benchmark of 23k shapes from 23 semantic object categories, annotated with 18 visual affordance categories. Based on this dataset, we provide three benchmarking tasks for evaluating visual affordance understanding, including full-shape, partial-view and rotation-invariant affordance estimations. Three state-of-the-art point cloud deep learning networks are evaluated on all tasks. In addition we also investigate a semi-supervised learning setup to explore the possibility to benefit from unlabeled data. Comprehensive results on our contributed dataset show the promise of visual affordance understanding as a valuable yet challenging benchmark.

READ FULL TEXT

page 4

page 7

page 13

page 14

page 15

page 16

page 17

page 18

research
05/02/2022

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding

Semantic understanding of 3D point cloud relies on learning models with ...
research
03/10/2023

MVImgNet: A Large-scale Dataset of Multi-view Images

Being data-driven is one of the most iconic properties of deep learning ...
research
08/28/2023

Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation

Visual bird's eye view (BEV) semantic segmentation helps autonomous vehi...
research
06/28/2021

Rail-5k: a Real-World Dataset for Rail Surface Defects Detection

This paper presents the Rail-5k dataset for benchmarking the performance...
research
06/11/2023

On the Efficacy of 3D Point Cloud Reinforcement Learning

Recent studies on visual reinforcement learning (visual RL) have explore...
research
07/18/2018

Visual Affordance and Function Understanding: A Survey

Nowadays, robots are dominating the manufacturing, entertainment and hea...
research
09/08/2021

YouRefIt: Embodied Reference Understanding with Language and Gesture

We study the understanding of embodied reference: One agent uses both la...

Please sign up or login with your details

Forgot password? Click here to reset