Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction

by   André Brás, et al.

Human gesture recognition has assumed a capital role in industrial applications, such as Human-Machine Interaction. We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features, which are drawn from the skeleton data provided by the Kinect sensor. The module for gesture detection relies on a feedforward neural network which performs framewise binary classification. The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions. Then we combine windows of varying durations to get a multi-temporal scale approach and an additional gain in performance. Encouraged by the recent success of Recurrent Neural Networks for time series domains, we also propose a method for simultaneous gesture segmentation and classification based on the bidirectional Long Short-Term Memory cells, which have shown ability for learning the temporal relationships on long temporal scales. We evaluate all the different approaches on the dataset published for the ChaLearn Looking at People Challenge 2014. The most effective method achieves a Jaccard index of 0.75, which suggests a performance almost on pair with that presented by the state-of-the-art techniques. At the end, the recognized gestures are used to interact with a collaborative robot.


Gesture Recognition with a Skeleton-Based Keyframe Selection Module

We propose a bidirectional consecutively connected two-pathway network (...

CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

Human activity and gesture recognition is an important component of rapi...

Deep Fisher Discriminant Learning for Mobile Hand Gesture Recognition

Gesture recognition is a challenging problem in the field of biometrics....

An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition

The focus of this paper is dynamic gesture recognition in the context of...

ChaLearn Looking at People: IsoGD and ConGD Large-scale RGB-D Gesture Recognition

The ChaLearn large-scale gesture recognition challenge has been run twic...

Gesture-based Human-Machine Interaction: Taxonomy, Problem Definition, and Analysis

The possibility for humans to interact with physical or virtual systems ...

Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks

This paper addresses the problem of continuous gesture recognition from ...