View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition

04/20/2018
by   Pengfei Zhang, et al.
0

Skeleton-based human action recognition has recently attracted increasing attention thanks to the accessibility and the popularity of 3D skeleton data. One of the key challenges in skeleton-based action recognition lies in the large view variations when capturing data. In order to alleviate the effects of view variations, this paper introduces a novel view adaptation scheme, which automatically determines the virtual observation viewpoints in a learning based data driven manner. We design two view adaptive neural networks, i.e., VA-RNN based on RNN, and VA-CNN based on CNN.. For each network, a novel view adaptation module learns and determines the most suitable observation viewpoints, and transforms the skeletons to those viewpoints for the end-to-end recognition with a main classification network. Ablation studies find that the proposed view adaptive models are capable of transforming the skeletons of various viewpoints to much more consistent virtual viewpoints which largely eliminates the viewpoint influence. In addition, we design a two-stream scheme (referred to as VA-fusion) that fuses the scores of the two networks to provide the fused prediction. Extensive experimental evaluations on five challenging benchmarks demonstrate that the effectiveness of the proposed view-adaptive networks and superior performance over state-of-the-art approaches.

READ FULL TEXT

page 1

page 11

page 14

research
03/24/2017

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Skeleton-based human action recognition has recently attracted increasin...
research
09/23/2022

View-Invariant Skeleton-based Action Recognition via Global-Local Contrastive Learning

Skeleton-based human action recognition has been drawing more interest r...
research
12/15/2019

Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks

Graph convolutional networks (GCNs), which generalize CNNs to more gener...
research
11/19/2021

Action Recognition with Domain Invariant Features of Skeleton Image

Due to the fast processing-speed and robustness it can achieve, skeleton...
research
02/14/2020

A Survey on 3D Skeleton-Based Action Recognition Using Learning Method

3D skeleton-based action recognition, owing to the latent advantages of ...
research
11/26/2020

Group-Skeleton-Based Human Action Recognition in Complex Events

Human action recognition as an important application of computer vision ...
research
03/28/2017

Learning and Refining of Privileged Information-based RNNs for Action Recognition from Depth Sequences

Existing RNN-based approaches for action recognition from depth sequence...

Please sign up or login with your details

Forgot password? Click here to reset