Enhancing Egocentric 3D Pose Estimation with Third Person Views

01/06/2022
by   Ameya Dhamanaskar, et al.
4

In this paper, we propose a novel approach to enhance the 3D body pose estimation of a person computed from videos captured from a single wearable camera. The key idea is to leverage high-level features linking first- and third-views in a joint embedding space. To learn such embedding space we introduce First2Third-Pose, a new paired synchronized dataset of nearly 2,000 videos depicting human activities captured from both first- and third-view perspectives. We explicitly consider spatial- and motion-domain features, combined using a semi-Siamese architecture trained in a self-supervised fashion. Experimental results demonstrate that the joint multi-view embedded space learned with our dataset is useful to extract discriminatory features from arbitrary single-view egocentric videos, without needing domain adaptation nor knowledge of camera parameters. We achieve significant improvement of egocentric 3D body pose estimation performance on two unconstrained datasets, over three supervised state-of-the-art approaches. Our dataset and code will be available for research purposes.

READ FULL TEXT

page 3

page 7

page 11

page 16

research
08/17/2021

Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

We present a self-supervised learning algorithm for 3D human pose estima...
research
04/06/2021

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

Existing approaches for multi-view multi-person 3D pose estimation expli...
research
02/02/2023

Hand Pose Estimation via Multiview Collaborative Self-Supervised Learning

3D hand pose estimation has made significant progress in recent years. H...
research
07/22/2022

My View is the Best View: Procedure Learning from Egocentric Videos

Procedure learning involves identifying the key-steps and determining th...
research
10/21/2022

3D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projections

3D human pose estimation in multi-view operating room (OR) videos is a r...
research
12/02/2019

View-Invariant Probabilistic Embedding for Human Pose

Depictions of similar human body configurations can vary with changing v...
research
03/11/2021

Robust 2D/3D Vehicle Parsing in CVIS

We present a novel approach to robustly detect and perceive vehicles in ...

Please sign up or login with your details

Forgot password? Click here to reset