DeepAI AI Chat
Log In Sign Up

In-Place Gestures Classification via Long-term Memory Augmented Network

by   Lizhi Zhao, et al.

In-place gesture-based virtual locomotion techniques enable users to control their viewpoint and intuitively move in the 3D virtual environment. A key research problem is to accurately and quickly recognize in-place gestures, since they can trigger specific movements of virtual viewpoints and enhance user experience. However, to achieve real-time experience, only short-term sensor sequence data (up to about 300ms, 6 to 10 frames) can be taken as input, which actually affects the classification performance due to limited spatio-temporal information. In this paper, we propose a novel long-term memory augmented network for in-place gestures classification. It takes as input both short-term gesture sequence samples and their corresponding long-term sequence samples that provide extra relevant spatio-temporal information in the training phase. We store long-term sequence features with an external memory queue. In addition, we design a memory augmented loss to help cluster features of the same class and push apart features from different classes, thus enabling our memory queue to memorize more relevant long-term sequence features. In the inference phase, we input only short-term sequence samples to recall the stored features accordingly, and fuse them together to predict the gesture class. We create a large-scale in-place gestures dataset from 25 participants with 11 gestures. Our method achieves a promising accuracy of 95.1 192ms, and an accuracy of 97.3 be superior to recent in-place gesture classification techniques. User study also validates our approach. Our source code and dataset will be made available to the community.


page 5

page 7


Egocentric Gesture Recognition for Head-Mounted AR devices

Natural interaction with virtual objects in AR/VR environments makes for...

Graph Tree Memory Networks

We introduce Graph Tree Memory Networks that memorize and remember any d...

Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction

Human gesture recognition has assumed a capital role in industrial appli...

Spatio-Temporal Representation Learning Enhanced Source Cell-phone Recognition from Speech Recordings

The existing source cell-phone recognition method lacks the long-term fe...

Effective Gesture Based Framework for Capturing User Input

Computers today aren't just confined to laptops and desktops. Mobile gad...

Classifying In-Place Gestures with End-to-End Point Cloud Learning

Walking in place for moving through virtual environments has attracted n...

Toward Inverse Control of Physics-Based Sound Synthesis

Long Short-Term Memory networks (LSTMs) can be trained to realize invers...