You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions

04/22/2019
by   Evonne Ng, et al.
0

The body pose of a person wearing a camera is of great interest for applications in augmented reality, healthcare, and robotics, yet much of the person's body is out of view for a typical wearable camera. We propose a learning-based approach to estimate the camera wearer's 3D body pose from egocentric video sequences. Our key insight is to leverage interactions with another person---whose body pose we can directly observe---as a signal inherently linked to the body pose of the first-person subject. We show that since interactions between individuals often induce a well-ordered series of back-and-forth responses, it is possible to learn a temporal model of the interlinked poses even though one party is largely out of view. We demonstrate our idea on a variety of domains with dyadic interaction and show the substantial impact on egocentric body pose estimation, which improves the state of the art. Video results are available at http://vision.cs.utexas.edu/projects/you2me/

READ FULL TEXT

page 1

page 2

page 4

page 5

page 8

research
03/24/2016

Seeing Invisible Poses: Estimating 3D Body Pose from Egocentric Video

Understanding the camera wearer's activity is central to egocentric visi...
research
12/01/2021

PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

Reconstructing the 3D pose of a person in metric scale from a single vie...
research
04/12/2021

Egocentric Pose Estimation from Human Vision Span

Estimating camera wearer's body pose from an egocentric view (egopose) i...
research
09/06/2022

Surya Namaskar: real-time advanced yoga pose recognition and correction for smart healthcare

Nowadays, yoga has gained worldwide attention because of increasing leve...
research
05/29/2017

Pose-Aware Person Recognition

Person recognition methods that use multiple body regions have shown sig...
research
03/22/2020

Pose to Seat: Automated Design of Body-Supporting Surfaces

The design of functional seating furniture is a complicated process whic...
research
11/01/2022

No-audio speaking status detection in crowded settings via visual pose-based filtering and wearable acceleration

Recognizing who is speaking in a crowded scene is a key challenge toward...

Please sign up or login with your details

Forgot password? Click here to reset