DeepAI AI Chat
Log In Sign Up

Motion Matters: Neural Motion Transfer for Better Camera Physiological Sensing

by   Akshay Paruchuri, et al.

Machine learning models for camera-based physiological measurement can have weak generalization due to a lack of representative training data. Body motion is one of the most significant sources of noise when attempting to recover the subtle cardiac pulse from a video. We explore motion transfer as a form of data augmentation to introduce motion variation while preserving physiological changes. We adapt a neural video synthesis approach to augment videos for the task of remote photoplethysmography (PPG) and study the effects of motion augmentation with respect to 1) the magnitude and 2) the type of motion. After training on motion-augmented versions of publicly available datasets, the presented inter-dataset results on five benchmark datasets show improvements of up to 75 utility of motion transfer as a data augmentation technique for improving the generalization of models for camera-based physiological sensing. We release our code and pre-trained models for using motion transfer as a data augmentation technique on our project page:


page 1

page 3

page 4

page 5


Empirical Study of Mix-based Data Augmentation Methods in Physiological Time Series Data

Data augmentation is a common practice to help generalization in the pro...

Federated Remote Physiological Measurement with Imperfect Data

The growing need for technology that supports remote healthcare is being...

The 1st Challenge on Remote Physiological Signal Sensing (RePSS)

Remote measurement of physiological signals from videos is an emerging t...

Higher Order of Motion Magnification for Vessel Localisation in Surgical Video

Locating vessels during surgery is critical for avoiding inadvertent dam...

SCAMPS: Synthetics for Camera Measurement of Physiological Signals

The use of cameras and computational algorithms for noninvasive, low-cos...

DeepMag: Source Specific Motion Magnification Using Gradient Ascent

Many important physical phenomena involve subtle signals that are diffic...

MetaPhys: Unsupervised Few-Shot Adaptation for Non-Contact Physiological Measurement

There are large individual differences in physiological processes, makin...