GCsT: Graph Convolutional Skeleton Transformer for Action Recognition

09/07/2021
by   Ruwen Bai, et al.
12

Graph convolutional networks (GCNs) achieve promising performance for skeleton-based action recognition. However, in most GCN-based methods, the spatial-temporal graph convolution is strictly restricted by the graph topology while only captures the short-term temporal context, thus lacking the flexibility of feature extraction. In this work, we present a novel architecture, named Graph Convolutional skeleton Transformer (GCsT), which addresses limitations in GCNs by introducing Transformer. Our GCsT employs all the benefits of Transformer (i.e. dynamical attention and global context) while keeps the advantages of GCNs (i.e. hierarchy and local topology structure). In GCsT, the spatial-temporal GCN forces the capture of local dependencies while Transformer dynamically extracts global spatial-temporal relationships. Furthermore, the proposed GCsT shows stronger expressive capability by adding additional information present in skeleton sequences. Incorporating the Transformer allows that information to be introduced into the model almost effortlessly. We validate the proposed GCsT by conducting extensive experiments, which achieves the state-of-the-art performance on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA datasets.

READ FULL TEXT

page 4

page 5

page 7

research
01/23/2018

Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Dynamics of human body skeletons convey significant information for huma...
research
01/27/2023

Skeleton-based Action Recognition through Contrasting Two-Stream Spatial-Temporal Networks

For pursuing accurate skeleton-based action recognition, most prior meth...
research
03/31/2022

SpatioTemporal Focus for Skeleton-based Action Recognition

Graph convolutional networks (GCNs) are widely adopted in skeleton-based...
research
10/06/2022

Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition

Despite great progress achieved by transformer in various vision tasks, ...
research
08/01/2022

Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer

Movement synchrony reflects the coordination of body movements between i...
research
10/26/2021

IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition

Recently, Transformer-based networks have shown great promise on skeleto...
research
08/03/2021

Skeleton Split Strategies for Spatial Temporal Graph Convolution Networks

A skeleton representation of the human body has been proven to be effect...

Please sign up or login with your details

Forgot password? Click here to reset