DeepBreath: Deep Learning of Breathing Patterns for Automatic Stress Recognition using Low-Cost Thermal Imaging in Unconstrained Settings

by   Youngjun Cho, et al.

We propose DeepBreath, a deep learning model which automatically recognises people's psychological stress level (mental overload) from their breathing patterns. Using a low cost thermal camera, we track a person's breathing patterns as temperature changes around his/her nostril. The paper's technical contribution is threefold. First of all, instead of creating hand-crafted features to capture aspects of the breathing patterns, we transform the uni-dimensional breathing signals into two dimensional respiration variability spectrogram (RVS) sequences. The spectrograms easily capture the complexity of the breathing dynamics. Second, a spatial pattern analysis based on a deep Convolutional Neural Network (CNN) is directly applied to the spectrogram sequences without the need of hand-crafting features. Finally, a data augmentation technique, inspired from solutions for over-fitting problems in deep learning, is applied to allow the CNN to learn with a small-scale dataset from short-term measurements (e.g., up to a few hours). The model is trained and tested with data collected from people exposed to two types of cognitive tasks (Stroop Colour Word Test, Mental Computation test) with sessions of different difficulty levels. Using normalised self-report as ground truth, the CNN reaches 84.59 56.52 outperformed powerful shallow learning methods based on a single layer neural network. Finally, the dataset of labelled thermal images will be open to the community.



There are no comments yet.


page 3

page 4

page 5


Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging

Background: A smartphone is a promising tool for daily cardiovascular me...

Nose Heat: Exploring Stress-induced Nasal Thermal Variability through Mobile Thermal Imaging

Automatically monitoring and quantifying stress-induced thermal dynamic ...

Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns

We introduce Deep Thermal Imaging, a new approach for close-range automa...

ThermSense: Smartphone-based Breathing Sensing Platform using Noncontact Low-Cost Thermal Camera

The ability of sensing breathing is becoming an increasingly important f...

Subject-Independent Drowsiness Recognition from Single-Channel EEG with an Interpretable CNN-LSTM model

For EEG-based drowsiness recognition, it is desirable to use subject-ind...

A Novel Deep Learning Method for Thermal to Annotated Thermal-Optical Fused Images

Thermal Images profile the passive radiation of objects and capture them...

Discovering outliers in the Mars Express thermal power consumption patterns

The Mars Express (MEX) spacecraft has been orbiting Mars since 2004. The...

Code Repositories


Mathematics Test as a mental stress-induction task (in Matlab): DeepBreath Open source

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.