CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

07/22/2020
by   Kenneth Lai, et al.
0

Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46 gesture-14/28 dataset.

READ FULL TEXT
08/10/2017

Motion Feature Augmented Recurrent Neural Network for Skeleton-based Dynamic Hand Gesture Recognition

Dynamic hand gesture recognition has attracted increasing interests beca...
08/13/2020

An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition

The focus of this paper is dynamic gesture recognition in the context of...
08/26/2020

Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction

Human gesture recognition has assumed a capital role in industrial appli...
08/22/2016

Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

This paper addresses the problem of continuous gesture recognition from ...
10/31/2017

RGB-D-based Human Motion Recognition with Deep Learning: A Survey

Human motion recognition is one of the most important branches of human-...
04/10/2019

Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation

With the advance of technologies, machines are increasingly present in p...
09/20/2021

Dynamic Gesture Recognition

The Human-Machine Interaction (HMI) research field is an important topic...