Hand-Object Interaction Reasoning

01/13/2022
by   Jian Ma, et al.
10

This paper proposes an interaction reasoning network for modelling spatio-temporal relationships between hands and objects in video. The proposed interaction unit utilises a Transformer module to reason about each acting hand, and its spatio-temporal relation to the other hand as well as objects being interacted with. We show that modelling two-handed interactions are critical for action recognition in egocentric video, and demonstrate that by using positionally-encoded trajectories, the network can better recognise observed interactions. We evaluate our proposal on EPIC-KITCHENS and Something-Else datasets, with an ablation study.

READ FULL TEXT

page 1

page 5

page 7

page 8

research
05/04/2023

Modelling Spatio-Temporal Interactions for Compositional Action Recognition

Humans have the natural ability to recognize actions even if the objects...
research
08/26/2021

Spatio-Temporal Dynamic Inference Network for Group Activity Recognition

Group activity recognition aims to understand the activity performed by ...
research
05/16/2022

TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement

We present TOCH, a method for refining incorrect 3D hand-object interact...
research
09/27/2016

Understanding and Exploiting Object Interaction Landscapes

Interactions play a key role in understanding objects and scenes, for bo...
research
12/21/2015

Harnessing the Deep Net Object Models for Enhancing Human Action Recognition

In this study, the influence of objects is investigated in the scenario ...
research
03/25/2019

Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph

Visual relationship reasoning is a crucial yet challenging task for unde...
research
07/15/2021

What and When to Look?: Temporal Span Proposal Network for Video Visual Relation Detection

Identifying relations between objects is central to understanding the sc...

Please sign up or login with your details

Forgot password? Click here to reset