Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

10/16/2016
by   Srinath Sridhar, et al.
0

Real-time simultaneous tracking of hands manipulating and interacting with external objects has many potential applications in augmented reality, tangible computing, and wearable computing. However, due to difficult occlusions, fast motions, and uniform hand appearance, jointly tracking hand and object pose is more challenging than tracking either of the two separately. Many previous approaches resort to complex multi-camera setups to remedy the occlusion problem and often employ expensive segmentation and optimization steps which makes real-time tracking impossible. In this paper, we propose a real-time solution that uses a single commodity RGB-D camera. The core of our approach is a 3D articulated Gaussian mixture alignment strategy tailored to hand-object tracking that allows fast pose optimization. The alignment energy uses novel regularizers to address occlusions and hand-object contacts. For added robustness, we guide the optimization with discriminative part classification of the hand and segmentation of the object. We conducted extensive experiments on several existing datasets and introduce a new annotated hand-object dataset. Quantitative and qualitative results show the key advantages of our method: speed, accuracy, and robustness.

READ FULL TEXT

page 2

page 5

page 12

page 13

page 14

research
04/07/2017

Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor

We present an approach for real-time, robust and accurate hand pose esti...
research
02/11/2016

Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model

Real-time marker-less hand tracking is of increasing importance in human...
research
02/12/2016

Fast and Robust Hand Tracking Using Detection-Guided Optimization

Markerless tracking of hands and fingers is a promising enabler for huma...
research
10/04/2015

Efficient Hand Articulations Tracking using Adaptive Hand Model and Depth map

Real-time hand articulations tracking is important for many applications...
research
07/05/2018

A Gauss-Newton Approach to Real-Time Monocular Multiple Object Tracking

We propose an algorithm for real-time 6DOF pose tracking of rigid 3D obj...
research
05/22/2017

Dynamics Based 3D Skeletal Hand Tracking

Tracking the full skeletal pose of the hands and fingers is a challengin...
research
08/22/2023

Affordance segmentation of hand-occluded containers from exocentric images

Visual affordance segmentation identifies the surfaces of an object an a...

Please sign up or login with your details

Forgot password? Click here to reset