A self-organizing neural network architecture for learning human-object interactions

10/05/2017
by   Luiza Mici, et al.
0

The visual recognition of transitive actions comprising human-object interactions is a key component enabling artificial systems to operate in natural environments. This challenging task requires, in addition to the recognition of articulated body actions, the extraction of semantic elements from the scene such as the identity of the manipulated objects. In this paper, we present a self-organizing neural network for the recognition of human-object interactions from RGB-D videos. Our model consists of a hierarchy of Grow When Required (GWR) networks which learn prototypical representations of body motion patterns and objects, also accounting for the development of action-object mappings in an unsupervised fashion. To demonstrate this ability, we report experimental results on a dataset of daily activities collected for the purpose of this study as well as on a publicly available benchmark dataset. In line with neurophysiological studies, our self-organizing architecture shows higher neural activation for congruent action-object pairs learned during training sessions with respect to artificially created incongruent ones. We show that our model achieves good classification accuracy on the benchmark dataset in an unsupervised fashion, showing competitive performance with respect to strictly supervised state-of-the-art approaches.

READ FULL TEXT
research
04/20/2022

THORN: Temporal Human-Object Relation Network for Action Recognition

Most action recognition models treat human activities as unitary events....
research
06/03/2019

How Much Does Audio Matter to Recognize Egocentric Object Interactions?

Sounds are an important source of information on our daily interactions ...
research
12/20/2019

Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks

Human action is naturally compositional: humans can easily recognize and...
research
08/22/2019

Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networks

Recognising human actions is a vital task for a humanoid robot, especial...
research
05/28/2018

Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization

Humans excel at continually acquiring and fine-tuning knowledge over sus...
research
01/04/2020

Human Action Recognition and Assessment via Deep Neural Network Self-Organization

The robust recognition and assessment of human actions are crucial in hu...
research
09/10/2019

Reasoning About Human-Object Interactions Through Dual Attention Networks

Objects are entities we act upon, where the functionality of an object i...

Please sign up or login with your details

Forgot password? Click here to reset