Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips

by   Panpan Zhang, et al.

Heart beat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature Generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The experimental results on the three datasets show that the proposed network outperforms the state-of-the-art methods.



There are no comments yet.


page 18

page 23

page 27


Recovering remote Photoplethysmograph Signal from Facial videos Using Spatio-Temporal Convolutional Networks

Recently average heart rate (HR) can be measured relatively accurately f...

Efficient Remote Photoplethysmography with Temporal Derivative Modules and Time-Shift Invariant Loss

We present a lightweight neural model for remote heart rate estimation f...

STC-Flow: Spatio-temporal Context-aware Optical Flow Estimation

In this paper, we propose a spatio-temporal contextual network, STC-Flow...

The Benefit of Distraction: Denoising Remote Vitals Measurements using Inverse Attention

Attention is a powerful concept in computer vision. End-to-end networks ...

Architectural Tricks for Deep Learning in Remote Photoplethysmography

Architectural improvements are studied for convolutional network perform...

StressNet: Detecting Stress in Thermal Videos

Precise measurement of physiological signals is critical for the effecti...

TransPPG: Two-stream Transformer for Remote Heart Rate Estimate

Non-contact facial video-based heart rate estimation using remote photop...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.