Preterm Infants’ Pose Estimation With Spatio-Temporal Features

by   efrontoni, et al.

Objective: Preterm infants’ limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants’ health status and motor/cognitive development. Herein, we propose a new approach to preterm infants’ limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability. Methods: Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset). Results: When applied to pose estimation, the median root mean square distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27 pixels). Conclusion: Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity). Significance: This article significantly enhances the state of art in automatic assessment of preterm infants’ health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants’ pose estimation.



page 1

page 2

page 3

page 4

page 5

page 8

page 9


Preterm infants' limb-pose estimation from depth images using convolutional neural networks

Preterm infants' limb-pose estimation is a crucial but challenging task,...

GAST-Net: Graph Attention Spatio-temporal Convolutional Networks for 3D Human Pose Estimation in Video

3D pose estimation in video can benefit greatly from both temporal and s...

Context-Aware Deep Spatio-Temporal Network for Hand Pose Estimation from Depth Images

As a fundamental and challenging problem in computer vision, hand pose e...

4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes

Tracking and localizing objects is a central problem in computer-assiste...

T-LEAP: occlusion-robust pose estimation of walking cows using temporal information

As herd size on dairy farms continue to increase, automatic health monit...

Multimodal Spatio-Temporal Deep Learning Approach for Neonatal Postoperative Pain Assessment

The current practice for assessing neonatal postoperative pain relies on...

Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation

Hand pose estimation is difficult due to different environmental conditi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.