3D Human Pose Lifting with Grid Convolution

02/17/2023
by   Yangyuxuan Kang, et al.
0

Existing lifting networks for regressing 3D human poses from 2D single-view poses are typically constructed with linear layers based on graph-structured representation learning. In sharp contrast to them, this paper presents Grid Convolution (GridConv), mimicking the wisdom of regular convolution operations in image space. GridConv is based on a novel Semantic Grid Transformation (SGT) which leverages a binary assignment matrix to map the irregular graph-structured human pose onto a regular weave-like grid pose representation joint by joint, enabling layer-wise feature learning with GridConv operations. We provide two ways to implement SGT, including handcrafted and learnable designs. Surprisingly, both designs turn out to achieve promising results and the learnable one is better, demonstrating the great potential of this new lifting representation learning formulation. To improve the ability of GridConv to encode contextual cues, we introduce an attention module over the convolutional kernel, making grid convolution operations input-dependent, spatial-aware and grid-specific. We show that our fully convolutional grid lifting network outperforms state-of-the-art methods with noticeable margins under (1) conventional evaluation on Human3.6M and (2) cross-evaluation on MPI-INF-3DHP. Code is available at https://github.com/OSVAI/GridConv

READ FULL TEXT

page 7

page 14

page 15

research
06/29/2023

MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling

Estimating 3D human poses only from a 2D human pose sequence is thorough...
research
02/20/2023

HTNet: Human Topology Aware Network for 3D Human Pose Estimation

3D human pose estimation errors would propagate along the human body top...
research
08/16/2021

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Human motion prediction is a challenging task due to the stochasticity a...
research
05/06/2021

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

Existing 3D human pose estimators suffer poor generalization performance...
research
08/05/2020

Pose-based Modular Network for Human-Object Interaction Detection

Human-object interaction(HOI) detection is a critical task in scene unde...
research
10/09/2021

Space-Time-Separable Graph Convolutional Network for Pose Forecasting

Human pose forecasting is a complex structured-data sequence-modelling t...
research
04/22/2021

Heterogeneous Grid Convolution for Adaptive, Efficient, and Controllable Computation

This paper proposes a novel heterogeneous grid convolution that builds a...

Please sign up or login with your details

Forgot password? Click here to reset