Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors

06/28/2021
by   Simon Bultmann, et al.
3

We present a novel method for estimation of 3D human poses from a multi-camera setup, employing distributed smart edge sensors coupled with a backend through a semantic feedback loop. 2D joint detection for each camera view is performed locally on a dedicated embedded inference processor. Only the semantic skeleton representation is transmitted over the network and raw images remain on the sensor board. 3D poses are recovered from 2D joints on a central backend, based on triangulation and a body model which incorporates prior knowledge of the human skeleton. A feedback channel from backend to individual sensors is implemented on a semantic level. The allocentric 3D pose is backprojected into the sensor views where it is fused with 2D joint detections. The local semantic model on each sensor can thus be improved by incorporating global context information. The whole pipeline is capable of real-time operation. We evaluate our method on three public datasets, where we achieve state-of-the-art results and show the benefits of our feedback architecture, as well as in our own setup for multi-person experiments. Using the feedback signal improves the 2D joint detections and in turn the estimated 3D poses.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
05/03/2022

3D Semantic Scene Perception using Distributed Smart Edge Sensors

We present a system for 3D semantic scene perception consisting of a net...
research
10/05/2021

Shape-aware Multi-Person Pose Estimation from Multi-View Images

In this paper we contribute a simple yet effective approach for estimati...
research
03/07/2023

External Camera-based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors

We present an approach for estimating a mobile robot's pose w.r.t. the a...
research
10/21/2022

3D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projections

3D human pose estimation in multi-view operating room (OR) videos is a r...
research
12/09/2019

DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image

In this paper, we propose a two-stage fully 3D network, namely DeepFuse,...
research
06/28/2021

Real-Time Human Pose Estimation on a Smart Walker using Convolutional Neural Networks

Rehabilitation is important to improve quality of life for mobility-impa...
research
10/27/2022

Robot to Human Object Handover using Vision and Joint Torque Sensor Modalities

We present a robot-to-human object handover algorithm and implement it o...

Please sign up or login with your details

Forgot password? Click here to reset