Statistical and Spatio-temporal Hand Gesture Features for Sign Language Recognition using the Leap Motion Sensor

02/22/2022
by   Jordan J. Bird, et al.
0

In modern society, people should not be identified based on their disability, rather, it is environments that can disable people with impairments. Improvements to automatic Sign Language Recognition (SLR) will lead to more enabling environments via digital technology. Many state-of-the-art approaches to SLR focus on the classification of static hand gestures, but communication is a temporal activity, which is reflected by many of the dynamic gestures present. Given this, temporal information during the delivery of a gesture is not often considered within SLR. The experiments in this work consider the problem of SL gesture recognition regarding how dynamic gestures change during their delivery, and this study aims to explore how single types of features as well as mixed features affect the classification ability of a machine learning model. 18 common gestures recorded via a Leap Motion Controller sensor provide a complex classification problem. Two sets of features are extracted from a 0.6 second time window, statistical descriptors and spatio-temporal attributes. Features from each set are compared by their ANOVA F-Scores and p-values, arranged into bins grown by 10 features per step to a limit of the 250 highest-ranked features. Results show that the best statistical model selected 240 features and scored 85.96 selected 230 features and scored 80.98 selected 240 features from each set leading to a classification accuracy of 86.75 learning models), the overall distribution shows that the minimum results are increased when inputs are any number of mixed features compared to any number of either of the two single sets of features.

READ FULL TEXT
research
03/28/2018

Exploiting Recurrent Neural Networks and Leap Motion Controller for Sign Language and Semaphoric Gesture Recognition

In human interactions, hands are a powerful way of expressing informatio...
research
06/23/2017

Multiresolution Match Kernels for Gesture Video Classification

The emergence of depth imaging technologies like the Microsoft Kinect ha...
research
08/24/2021

Real-time Indian Sign Language (ISL) Recognition

This paper presents a system which can recognise hand poses gestures...
research
09/30/2022

Deep Learning Classification of Touch Gestures Using Distributed Normal and Shear Force

When humans socially interact with another agent (e.g., human, pet, or r...
research
04/10/2019

Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation

With the advance of technologies, machines are increasingly present in p...
research
12/11/2018

The Impact of Quantity of Training Data on Recognition of Eating Gestures

This paper considers the problem of recognizing eating gestures by track...
research
01/02/2019

Adaptive EMG-based hand gesture recognition using hyperdimensional computing

Accurate recognition of hand gestures is crucial to the functionality of...

Please sign up or login with your details

Forgot password? Click here to reset