Graph Edge Convolutional Neural Networks for Skeleton Based Action Recognition

05/16/2018
by   Xikun Zhang, et al.
2

This paper investigates body bones from skeleton data for skeleton based action recognition. Body joints, as the direct result of mature pose estimation technologies, are always the key concerns of traditional action recognition methods. However, instead of joints, we humans naturally identify how the human body moves according to shapes, lengths and places of bones, which are more obvious and stable for observation. Hence given graphs generated from skeleton data, we propose to develop convolutions over graph edges that correspond to bones in human skeleton. We describe an edge by integrating its spatial neighboring edges to explore the cooperation between different bones, as well as its temporal neighboring edges to address the consistency of movements in an action. A graph edge convolutional neural network is then designed for skeleton based action recognition. Considering the complementarity between graph node convolution and graph edge convolution, we additionally construct two hybrid neural networks to combine graph node convolutional neural network and graph edge convolutional neural network using shared intermediate layers. Experimental results on Kinetics and NTU-RGB+D datasets demonstrate that our graph edge convolution is effective to capture characteristic of actions and our graph edge convolutional neural network significantly outperforms existing state-of-art skeleton based action recognition methods. Additionally, more performance improvements can be achieved by the hybrid networks.

READ FULL TEXT

page 14

page 15

page 18

research
03/17/2020

Predictively Encoded Graph Convolutional Network for Noise-Robust Skeleton-based Action Recognition

In skeleton-based action recognition, graph convolutional networks (GCNs...
research
11/29/2018

Generalized Graph Convolutional Networks for Skeleton-based Action Recognition

With the prevalence of accessible depth sensors, dynamic human body skel...
research
01/15/2022

Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance

Action recognition is a key algorithmic part of emerging on-the-edge sma...
research
11/12/2019

Deep-Aligned Convolutional Neural Network for Skeleton-based Action Recognition and Segmentation

Convolutional neural networks (CNNs) are deep learning frameworks which ...
research
01/31/2023

Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)

Recently, there has been a remarkable increase in the interest towards s...
research
01/08/2018

Ensemble One-dimensional Convolution Neural Networks for Skeleton-based Action Recognition

In this paper, we proposed a effective but extensible residual one-dimen...

Please sign up or login with your details

Forgot password? Click here to reset