CPF: Learning a Contact Potential Field to Model the Hand-object Interaction

by   Lixin Yang, et al.

Estimating hand-object (HO) pose during interaction has been brought remarkable growth in virtue of deep learning methods. Modeling the contact between the hand and object properly is the key to construct a plausible grasp. Yet, previous works usually focus on jointly estimating HO pose but not fully explore the physical contact preserved in grasping. In this paper, we present an explicit contact representation, Contact Potential Field (CPF) that models each hand-object contact as a spring-mass system. Then we can refine a natural grasp by minimizing the elastic energy w.r.t those systems. To recover CPF, we also propose a learning-fitting hybrid framework named MIHO. Extensive experiments on two public benchmarks have shown that our method can achieve state-of-the-art in several reconstruction metrics, and allow us to produce more physically plausible HO pose even when the ground-truth exhibits severe interpenetration or disjointedness. Our code is available at https://github.com/lixiny/CPF .


page 4

page 7

page 8

page 11

page 12

page 14

page 16


ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

Grasping is natural for humans. However, it involves complex hand config...

ContactOpt: Optimizing Contact to Improve Grasps

Physical contact between hands and objects plays a critical role in huma...

Learning joint reconstruction of hands and manipulated objects

Estimating hand-object manipulations is essential for interpreting and i...

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

We introduce the dynamic grasp synthesis task: given an object with a kn...

TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement

We present TOCH, a method for refining incorrect 3D hand-object interact...

Stability-driven Contact Reconstruction From Monocular Color Images

Physical contact provides additional constraints for hand-object state r...

Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction

Every hand-object interaction begins with contact. Despite predicting th...