CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

by   Kenneth Lai, et al.

Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46 gesture-14/28 dataset.


Motion Feature Augmented Recurrent Neural Network for Skeleton-based Dynamic Hand Gesture Recognition

Dynamic hand gesture recognition has attracted increasing interests beca...

An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition

The focus of this paper is dynamic gesture recognition in the context of...

Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction

Human gesture recognition has assumed a capital role in industrial appli...

Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

This paper addresses the problem of continuous gesture recognition from ...

RGB-D-based Human Motion Recognition with Deep Learning: A Survey

Human motion recognition is one of the most important branches of human-...

Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation

With the advance of technologies, machines are increasingly present in p...

Dynamic Gesture Recognition

The Human-Machine Interaction (HMI) research field is an important topic...