Classifying In-Place Gestures with End-to-End Point Cloud Learning

by   Lizhi Zhao, et al.

Walking in place for moving through virtual environments has attracted noticeable attention recently. Recent attempts focused on training a classifier to recognize certain patterns of gestures (e.g., standing, walking, etc) with the use of neural networks like CNN or LSTM. Nevertheless, they often consider very few types of gestures and/or induce less desired latency in virtual environments. In this paper, we propose a novel framework for accurate and efficient classification of in-place gestures. Our key idea is to treat several consecutive frames as a "point cloud". The HMD and two VIVE trackers provide three points in each frame, with each point consisting of 12-dimensional features (i.e., three-dimensional position coordinates, velocity, rotation, angular velocity). We create a dataset consisting of 9 gesture classes for virtual in-place locomotion. In addition to the supervised point-based network, we also take unsupervised domain adaptation into account due to inter-person variations. To this end, we develop an end-to-end joint framework involving both a supervised loss for supervised point learning and an unsupervised loss for unsupervised domain adaptation. Experiments demonstrate that our approach generates very promising outcomes, in terms of high overall classification accuracy (95.0 publicly available at:


page 3

page 8


Spatiotemporal Learning of Dynamic Gestures from 3D Point Cloud Data

In this paper, we demonstrate an end-to-end spatiotemporal gesture learn...

vLPD-Net: A Registration-aided Domain Adaptation Network for 3D Point Cloud Based Place Recognition

In the field of large-scale SLAM for autonomous driving and mobile robot...

Self-Distillation for Unsupervised 3D Domain Adaptation

Point cloud classification is a popular task in 3D vision. However, prev...

Self-Ensemling for 3D Point Cloud Domain Adaption

Recently 3D point cloud learning has been a hot topic in computer vision...

In-Place Gestures Classification via Long-term Memory Augmented Network

In-place gesture-based virtual locomotion techniques enable users to con...

Copy-Move Forgery Classification via Unsupervised Domain Adaptation

In the current era, image manipulation is becoming increasingly easier, ...

Please sign up or login with your details

Forgot password? Click here to reset