Video-based Remote Physiological Measurement via Self-supervised Learning

10/27/2022
by   Zijie Yue, et al.
0

Video-based remote physiological measurement aims to estimate remote photoplethysmography (rPPG) signals from human face videos and then measure multiple vital signs (e.g. heart rate, respiration frequency) from rPPG signals. Recent approaches achieve it by training deep neural networks, which normally require abundant face videos and synchronously recorded photoplethysmography (PPG) signals for supervision. However, the collection of these annotated corpora is uneasy in practice. In this paper, we introduce a novel frequency-inspired self-supervised framework that learns to estimate rPPG signals from face videos without the need of ground truth PPG signals. Given a video sample, we first augment it into multiple positive/negative samples which contain similar/dissimilar signal frequencies to the original one. Specifically, positive samples are generated using spatial augmentation. Negative samples are generated via a learnable frequency augmentation module, which performs non-linear signal frequency transformation on the input without excessively changing its visual appearance. Next, we introduce a local rPPG expert aggregation module to estimate rPPG signals from augmented samples. It encodes complementary pulsation information from different face regions and aggregate them into one rPPG prediction. Finally, we propose a series of frequency-inspired losses, i.e. frequency contrastive loss, frequency ratio consistency loss, and cross-video frequency agreement loss, for the optimization of estimated rPPG signals from multiple augmented video samples and across temporally neighboring video samples. We conduct rPPG-based heart rate, heart rate variability and respiration frequency estimation on four standard benchmarks. The experimental results demonstrate that our method improves the state of the art by a large margin.

READ FULL TEXT

page 1

page 3

research
08/08/2022

Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

Video-based remote physiological measurement utilizes face videos to mea...
research
07/16/2020

Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling

Remote physiological measurements, e.g., remote photoplethysmography (rP...
research
06/10/2021

Overcoming Difficulty in Obtaining Dark-skinned Subjects for Remote-PPG by Synthetic Augmentation

Camera-based remote photoplethysmography (rPPG) provides a non-contact w...
research
07/14/2020

Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner

Remote heart rate estimation is the measurement of heart rate without an...
research
11/18/2021

The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video

The ability to reliably estimate physiological signals from video is a p...
research
07/16/2021

Self-Supervised Learning Framework for Remote Heart Rate Estimation Using Spatiotemporal Augmentation

Recent supervised deep learning methods have shown that heart rate can b...
research
10/14/2020

The Benefit of Distraction: Denoising Remote Vitals Measurements using Inverse Attention

Attention is a powerful concept in computer vision. End-to-end networks ...

Please sign up or login with your details

Forgot password? Click here to reset