Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation

09/29/2021
by   Yao Lu, et al.
4

In this paper, we address the problem of estimating the hand pose from the egocentric view when the hand is interacting with objects. Specifically, we propose a method to label a dataset Ego-Siam which contains the egocentric images pair-wisely. We also use the collected pairwise data to train our encoder-decoder style network which has been proven efficient in. This could bring extra training efficiency and testing accuracy. Our network is lightweight and can be performed with over 30 FPS with an outdated GPU. We demonstrate that our method outperforms Mueller et al. which is the state of the art work dealing with egocentric hand-object interaction problems on the GANerated dataset. To show the ability to preserve the semantic information of our method, we also report the performance of grasp type classification on GUN-71 dataset and outperforms the benchmark by only using the predicted 3-d hand pose.

READ FULL TEXT

page 1

page 3

page 6

page 7

research
11/16/2022

Egocentric Hand-object Interaction Detection

In this paper, we propose a method to jointly determine the status of ha...
research
04/25/2021

Parallel mesh reconstruction streams for pose estimation of interacting hands

We present a new multi-stream 3D mesh reconstruction network (MSMR-Net) ...
research
08/22/2023

Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views

Hand-object interaction understanding and the barely addressed novel vie...
research
09/29/2021

Egocentric Hand-object Interaction Detection and Application

In this paper, we present a method to detect the hand-object interaction...
research
11/12/2019

HMTNet:3D Hand Pose Estimation from Single Depth Image Based on Hand Morphological Topology

Thanks to the rapid development of CNNs and depth sensors, great progres...
research
08/02/2022

In-Hand Pose Estimation and Pin Inspection for Insertion of Through-Hole Components

The insertion of through-hole components is a difficult task. As the tol...
research
02/23/2018

6D Pose Estimation using an Improved Method based on Point Pair Features

The Point Pair Feature (Drost et al. 2010) has been one of the most succ...

Please sign up or login with your details

Forgot password? Click here to reset