ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

09/12/2021
by   Kailin Li, et al.
5

Estimating the articulated 3D hand-object pose from a single RGB image is a highly ambiguous and challenging problem requiring large-scale datasets that contain diverse hand poses, object poses, and camera viewpoints. Most real-world datasets lack this diversity. In contrast, synthetic datasets can easily ensure vast diversity, but learning from them is inefficient and suffers from heavy training consumption. To address the above issues, we propose ArtiBoost, a lightweight online data enrichment method that boosts articulated hand-object pose estimation from the data perspective. ArtiBoost is employed along with a real-world source dataset. During training, ArtiBoost alternatively performs data exploration and synthesis. ArtiBoost can cover various hand-object poses and camera viewpoints based on a Compositional hand-object Configuration and Viewpoint space (CCV-space) and can adaptively enrich the current hard-discernable samples by a mining strategy. We apply ArtiBoost on a simple learning baseline network and demonstrate the performance boost on several hand-object benchmarks. As an illustrative example, with ArtiBoost, even a simple baseline network can outperform the previous start-of-the-art based on Transformer on the HO3D dataset. Our code is available at https://github.com/MVIG-SJTU/ArtiBoost.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 8

page 9

page 11

07/02/2019

HO-3D: A Multi-User, Multi-Object Dataset for Joint 3D Hand-Object Pose Estimation

We propose a new dataset for 3D hand+object pose estimation from color i...
08/21/2020

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

Analysis of hand-hand interactions is a crucial step towards better unde...
03/07/2020

Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands

Many manipulation tasks, such as placement or within-hand manipulation, ...
10/02/2020

MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis

Estimating the 3D hand pose from a monocular RGB image is important but ...
07/16/2022

NeFSAC: Neurally Filtered Minimal Samples

Since RANSAC, a great deal of research has been devoted to improving bot...
12/03/2018

Disentangling Latent Hands for Image Synthesis and Pose Estimation

Hand image synthesis and pose estimation from RGB images are both highly...

Code Repositories

ArtiBoost

Code for "ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis"


view repo