Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips

04/06/2021
by   Panpan Zhang, et al.
0

Heart beat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature Generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The experimental results on the three datasets show that the proposed network outperforms the state-of-the-art methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 18

page 23

page 27

05/07/2019

Recovering remote Photoplethysmograph Signal from Facial videos Using Spatio-Temporal Convolutional Networks

Recently average heart rate (HR) can be measured relatively accurately f...
03/21/2022

Efficient Remote Photoplethysmography with Temporal Derivative Modules and Time-Shift Invariant Loss

We present a lightweight neural model for remote heart rate estimation f...
03/01/2020

STC-Flow: Spatio-temporal Context-aware Optical Flow Estimation

In this paper, we propose a spatio-temporal contextual network, STC-Flow...
10/14/2020

The Benefit of Distraction: Denoising Remote Vitals Measurements using Inverse Attention

Attention is a powerful concept in computer vision. End-to-end networks ...
11/06/2019

Architectural Tricks for Deep Learning in Remote Photoplethysmography

Architectural improvements are studied for convolutional network perform...
11/18/2020

StressNet: Detecting Stress in Thermal Videos

Precise measurement of physiological signals is critical for the effecti...
01/26/2022

TransPPG: Two-stream Transformer for Remote Heart Rate Estimate

Non-contact facial video-based heart rate estimation using remote photop...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.