DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image

12/09/2019
by   Fuyang Huang, et al.
0

In this paper, we propose a two-stage fully 3D network, namely DeepFuse, to estimate human pose in 3D space by fusing body-worn Inertial Measurement Unit (IMU) data and multi-view images deeply. The first stage is designed for pure vision estimation. To preserve data primitiveness of multi-view inputs, the vision stage uses multi-channel volume as data representation and 3D soft-argmax as activation layer. The second one is the IMU refinement stage which introduces an IMU-bone layer to fuse the IMU and vision data earlier at data level. without requiring a given skeleton model a priori, we can achieve a mean joint error of 28.9mm on TotalCapture dataset and 13.4mm on Human3.6M dataset under protocol 1, improving the SOTA result by a large margin. Finally, we discuss the effectiveness of a fully 3D network for 3D pose estimation experimentally which may benefit future research.

READ FULL TEXT

page 4

page 6

page 7

research
01/25/2017

A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms

Many approaches have been proposed for human pose estimation in single a...
research
07/07/2021

PoseRN: A 2D pose refinement network for bias-free multi-view 3D human pose estimation

We propose a new 2D pose refinement network that learns to predict the h...
research
10/11/2022

ACRNet: Attention Cube Regression Network for Multi-view Real-time 3D Human Pose Estimation in Telemedicine

Human pose estimation (HPE) for 3D skeleton reconstruction in telemedici...
research
07/12/2023

Deep learning-based estimation of whole-body kinematics from multi-view images

It is necessary to analyze the whole-body kinematics (including joint lo...
research
01/07/2020

Deep Reinforcement Learning for Active Human Pose Estimation

Most 3d human pose estimation methods assume that input – be it images o...
research
06/28/2021

Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors

We present a novel method for estimation of 3D human poses from a multi-...
research
08/10/2017

Joint Multi-Person Pose Estimation and Semantic Part Segmentation

Human pose estimation and semantic part segmentation are two complementa...

Please sign up or login with your details

Forgot password? Click here to reset