Non-contact Pain Recognition from Video Sequences with Remote Physiological Measurements Prediction

05/18/2021
by   Ruijing Yang, et al.
0

Automatic pain recognition is paramount for medical diagnosis and treatment. The existing works fall into three categories: assessing facial appearance changes, exploiting physiological cues, or fusing them in a multi-modal manner. However, (1) appearance changes are easily affected by subjective factors which impedes objective pain recognition. Besides, the appearance-based approaches ignore long-range spatial-temporal dependencies that are important for modeling expressions over time; (2) the physiological cues are obtained by attaching sensors on human body, which is inconvenient and uncomfortable. In this paper, we present a novel multi-task learning framework which encodes both appearance changes and physiological cues in a non-contact manner for pain recognition. The framework is able to capture both local and long-range dependencies via the proposed attention mechanism for the learned appearance representations, which are further enriched by temporally attended physiological cues (remote photoplethysmography, rPPG) that are recovered from videos in the auxiliary task. This framework is dubbed rPPG-enriched Spatio-Temporal Attention Network (rSTAN) and allows us to establish the state-of-the-art performance of non-contact pain recognition on publicly available pain databases. It demonstrates that rPPG predictions can be used as an auxiliary task to facilitate non-contact automatic pain recognition.

READ FULL TEXT
research
08/10/2022

Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues

Face anti-spoofing (FAS) and face forgery detection play vital roles in ...
research
08/19/2021

Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

For a given video-based Human-Object Interaction scene, modeling the spa...
research
03/21/2023

BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurements

Understanding of human visual perception has historically inspired the d...
research
02/07/2023

PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer

Remote photoplethysmography (rPPG), which aims at measuring heart activi...
research
06/09/2022

Depression Recognition using Remote Photoplethysmography from Facial Videos

Depression is a mental illness that may be harmful to an individual's he...
research
08/15/2023

Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos

Remote photoplethysmography (rPPG) based physiological measurement is an...
research
04/16/2020

A Local Descriptor with Physiological Characteristic for Finger Vein Recognition

Local feature descriptors exhibit great superiority in finger vein recog...

Please sign up or login with your details

Forgot password? Click here to reset