Synthetic Video Generation for Robust Hand Gesture Recognition in Augmented Reality Applications

by   Varun Jain, et al.

Hand gestures are a natural means of interaction in Augmented Reality and Virtual Reality (AR/VR) applications. Recently, there has been an increased focus on removing the dependence of accurate hand gesture recognition on complex sensor setup found in expensive proprietary devices such as the Microsoft HoloLens, Daqri and Meta Glasses. Most such solutions either rely on multi-modal sensor data or deep neural networks that can benefit greatly from abundance of labelled data. Datasets are an integral part of any deep learning based research. They have been the principal reason for the substantial progress in this field, both, in terms of providing enough data for the training of these models, and, for benchmarking competing algorithms. However, it is becoming increasingly difficult to generate enough labelled data for complex tasks such as hand gesture recognition. The goal of this work is to introduce a framework capable of generating photo-realistic videos that have labelled hand bounding box and fingertip that can help in designing, training, and benchmarking models for hand-gesture recognition in AR/VR applications. We demonstrate the efficacy of our framework in generating videos with diverse backgrounds.


page 2

page 3


Hand-Gesture-Recognition Based Text Input Method for AR/VR Wearable Devices

Static and dynamic hand movements are basic way for human-machine intera...

Holoscopic 3D Micro-Gesture Database for Wearable Device Interaction

With the rapid development of augmented reality (AR) and virtual reality...

Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning

Human-Computer Interaction (HCI) has been the subject of research for ma...

LE-HGR: A Lightweight and Efficient RGB-based Online Gesture Recognition Network for Embedded AR Devices

Online hand gesture recognition (HGR) techniques are essential in augmen...

Real-time Egocentric Gesture Recognition on Mobile Head Mounted Displays

Mobile virtual reality (VR) head mounted displays (HMD) have become popu...

Improved Static Hand Gesture Classification on Deep Convolutional Neural Networks using Novel Sterile Training Technique

In this paper, we investigate novel data collection and training techniq...

Automated Training and Maintenance through Kinect

In this paper, we have worked on reducing burden on mechanic involving c...

Please sign up or login with your details

Forgot password? Click here to reset