DeepAI AI Chat
Log In Sign Up

Egocentric Gesture Recognition for Head-Mounted AR devices

by   Tejo Chalasani, et al.
Trinity College Dublin

Natural interaction with virtual objects in AR/VR environments makes for a smooth user experience. Gestures are a natural extension from real world to augmented space to achieve these interactions. Finding discriminating spatio-temporal features relevant to gestures and hands in ego-view is the primary challenge for recognising egocentric gestures. In this work we propose a data driven end-to-end deep learning approach to address the problem of egocentric gesture recognition, which combines an ego-hand encoder network to find ego-hand features, and a recurrent neural network to discern temporally discriminating features. Since deep learning networks are data intensive, we propose a novel data augmentation technique using green screen capture to alleviate the problem of ground truth annotation. In addition we publish a dataset of 10 gestures performed in a natural fashion in front of a green screen for training and the same 10 gestures performed in different natural scenes without green screen for validation. We also present the results of our network's performance in comparison to the state-of-the-art using the AirGest dataset


page 3

page 5


Simultaneous Segmentation and Recognition: Towards more accurate Ego Gesture Recognition

Ego hand gestures can be used as an interface in AR and VR environments....

Learning to recognize touch gestures: recurrent vs. convolutional features and dynamic sampling

We propose a fully automatic method for learning gestures on big touch d...

In-Place Gestures Classification via Long-term Memory Augmented Network

In-place gesture-based virtual locomotion techniques enable users to con...

Efficient Gesture Recognition for the Assistance of Visually Impaired People using Multi-Head Neural Networks

This paper proposes an interactive system for mobile devices controlled ...

CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition

Egocentric gestures are the most natural form of communication for human...

WristSketcher: Creating Dynamic Sketches in AR with a Sensing Wristband

Restricted by the limited interaction area of native AR glasses (e.g., t...

Spatiotemporal Learning of Dynamic Gestures from 3D Point Cloud Data

In this paper, we demonstrate an end-to-end spatiotemporal gesture learn...