In-Place Gestures Classification via Long-term Memory Augmented Network

09/02/2022
by   Lizhi Zhao, et al.
0

In-place gesture-based virtual locomotion techniques enable users to control their viewpoint and intuitively move in the 3D virtual environment. A key research problem is to accurately and quickly recognize in-place gestures, since they can trigger specific movements of virtual viewpoints and enhance user experience. However, to achieve real-time experience, only short-term sensor sequence data (up to about 300ms, 6 to 10 frames) can be taken as input, which actually affects the classification performance due to limited spatio-temporal information. In this paper, we propose a novel long-term memory augmented network for in-place gestures classification. It takes as input both short-term gesture sequence samples and their corresponding long-term sequence samples that provide extra relevant spatio-temporal information in the training phase. We store long-term sequence features with an external memory queue. In addition, we design a memory augmented loss to help cluster features of the same class and push apart features from different classes, thus enabling our memory queue to memorize more relevant long-term sequence features. In the inference phase, we input only short-term sequence samples to recall the stored features accordingly, and fuse them together to predict the gesture class. We create a large-scale in-place gestures dataset from 25 participants with 11 gestures. Our method achieves a promising accuracy of 95.1 192ms, and an accuracy of 97.3 be superior to recent in-place gesture classification techniques. User study also validates our approach. Our source code and dataset will be made available to the community.

READ FULL TEXT

page 5

page 7

research
08/16/2018

Egocentric Gesture Recognition for Head-Mounted AR devices

Natural interaction with virtual objects in AR/VR environments makes for...
research
11/03/2021

Graph Tree Memory Networks

We introduce Graph Tree Memory Networks that memorize and remember any d...
research
08/26/2020

Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction

Human gesture recognition has assumed a capital role in industrial appli...
research
08/25/2022

Spatio-Temporal Representation Learning Enhanced Source Cell-phone Recognition from Speech Recordings

The existing source cell-phone recognition method lacks the long-term fe...
research
08/01/2022

Effective Gesture Based Framework for Capturing User Input

Computers today aren't just confined to laptops and desktops. Mobile gad...
research
08/22/2021

Classifying In-Place Gestures with End-to-End Point Cloud Learning

Walking in place for moving through virtual environments has attracted n...
research
06/29/2017

Toward Inverse Control of Physics-Based Sound Synthesis

Long Short-Term Memory networks (LSTMs) can be trained to realize invers...

Please sign up or login with your details

Forgot password? Click here to reset