Efficient Multi-stream Temporal Learning and Post-fusion Strategy for 3D Skeleton-based Hand Activity Recognition

02/20/2021
by   Yasser Boutaleb, et al.
0

Recognizing first-person hand activity is a challenging task, especially when not enough data are available. In this paper, we tackle this challenge by proposing a new hybrid learning pipeline for skeleton-based hand activity recognition, which is composed of three blocks. First, for a given sequence of hand’s joint positions, the spatial features are extracted using a dedicated combination of local and global spatial hand-crafted features. Then, the temporal dependencies are learned using a multi-stream learning strategy. Finally, a hand activity sequence classifier is learned, via our Post-fusion strategy, applied to the previously learned temporal dependencies. The experiments, evaluated on two real-world data sets, show that our approach performs better than the state-of-the-art approaches. For more ablation studies, we compared our Post-fusion strategy with three traditional fusion baselines and showed an improvement above 2.4% of accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/01/2022

Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition

First-person hand activity recognition is a challenging task, especially...
research
02/01/2016

Combining ConvNets with Hand-Crafted Features for Action Recognition Based on an HMM-SVM Classifier

This paper proposes a new framework for RGB-D-based action recognition t...
research
04/29/2019

A neural network based on SPD manifold learning for skeleton-based hand gesture recognition

This paper proposes a new neural network based on SPD manifold learning ...
research
05/05/2023

Distilled Mid-Fusion Transformer Networks for Multi-Modal Human Activity Recognition

Human Activity Recognition is an important task in many human-computer c...
research
11/10/2022

SWTF: Sparse Weighted Temporal Fusion for Drone-Based Activity Recognition

Drone-camera based human activity recognition (HAR) has received signifi...
research
10/25/2018

HAR-Net:Fusing Deep Representation and Hand-crafted Features for Human Activity Recognition

Wearable computing and context awareness are the focuses of study in the...
research
11/01/2018

Skeleton-based Activity Recognition with Local Order Preserving Match of Linear Patches

Human activity recognition has drawn considerable attention recently in ...

Please sign up or login with your details

Forgot password? Click here to reset