DeepAI AI Chat
Log In Sign Up

Precise Affordance Annotation for Egocentric Action Video Datasets

by   Zecheng Yu, et al.
The University of Tokyo

Object affordance is an important concept in human-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning. However, existing datasets often: 1) mix up affordance with object functionality; 2) confuse affordance with goal-related action; and 3) ignore human motor capacity. This paper proposes an efficient annotation scheme to address these issues by combining goal-irrelevant motor actions and grasp types as affordance labels and introducing the concept of mechanical action to represent the action possibilities between two objects. We provide new annotations by applying this scheme to the EPIC-KITCHENS dataset and test our annotation with tasks such as affordance recognition. We qualitatively verify that models trained with our annotation can distinguish affordance and mechanical actions.


page 3

page 4


Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videos

Object affordance is an important concept in hand-object interaction, pr...

Improving Human Action Recognition by Non-action Classification

In this paper we consider the task of recognizing human actions in reali...

Learning Motor Resonance in Human-Human and Human-Robot Interaction with Coupled Dynamical System

Human interaction involves very sophisticated non-verbal communication s...

Robot peels banana with goal-conditioned dual-action deep imitation learning

A long-horizon dexterous robot manipulation task of deformable objects, ...

Forecasting Human Object Interaction: Joint Prediction of Motor Attention and Egocentric Activity

We address the challenging task of anticipating human-object interaction...

Reasoning About Human-Object Interactions Through Dual Attention Networks

Objects are entities we act upon, where the functionality of an object i...

Actionable Email Intent Modeling with Reparametrized RNNs

Emails in the workplace are often intentional calls to action for its re...