DeepAI AI Chat
Log In Sign Up

Classifying In-Place Gestures with End-to-End Point Cloud Learning

by   Lizhi Zhao, et al.

Walking in place for moving through virtual environments has attracted noticeable attention recently. Recent attempts focused on training a classifier to recognize certain patterns of gestures (e.g., standing, walking, etc) with the use of neural networks like CNN or LSTM. Nevertheless, they often consider very few types of gestures and/or induce less desired latency in virtual environments. In this paper, we propose a novel framework for accurate and efficient classification of in-place gestures. Our key idea is to treat several consecutive frames as a "point cloud". The HMD and two VIVE trackers provide three points in each frame, with each point consisting of 12-dimensional features (i.e., three-dimensional position coordinates, velocity, rotation, angular velocity). We create a dataset consisting of 9 gesture classes for virtual in-place locomotion. In addition to the supervised point-based network, we also take unsupervised domain adaptation into account due to inter-person variations. To this end, we develop an end-to-end joint framework involving both a supervised loss for supervised point learning and an unsupervised loss for unsupervised domain adaptation. Experiments demonstrate that our approach generates very promising outcomes, in terms of high overall classification accuracy (95.0 publicly available at:


page 3

page 8


Spatiotemporal Learning of Dynamic Gestures from 3D Point Cloud Data

In this paper, we demonstrate an end-to-end spatiotemporal gesture learn...

vLPD-Net: A Registration-aided Domain Adaptation Network for 3D Point Cloud Based Place Recognition

In the field of large-scale SLAM for autonomous driving and mobile robot...

Self-Distillation for Unsupervised 3D Domain Adaptation

Point cloud classification is a popular task in 3D vision. However, prev...

In-Place Gestures Classification via Long-term Memory Augmented Network

In-place gesture-based virtual locomotion techniques enable users to con...

CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation

3D LiDAR semantic segmentation is fundamental for autonomous driving. Se...

1st Place Solution for Waymo Open Dataset Challenge – 3D Detection and Domain Adaptation

In this technical report, we introduce our winning solution "HorizonLiDA...

Code Repositories


Code for paper "Classifying In-Place Gestures with End-to-End Point Cloud Learning"

view repo