Identification of Cognitive Workload during Surgical Tasks with Multimodal Deep Learning

by   Kaizhe Jin, et al.

The operating room (OR) is a dynamic and complex environment consisting of a multidisciplinary team working together in a high take environment to provide safe and efficient patient care. Additionally, surgeons are frequently exposed to multiple psycho-organisational stressors that may cause negative repercussions on their immediate technical performance and long-term health. Many factors can therefore contribute to increasing the Cognitive Workload (CWL) such as temporal pressures, unfamiliar anatomy or distractions in the OR. In this paper, a cascade of two machine learning approaches is suggested for the multimodal recognition of CWL in four different surgical task conditions. Firstly, a model based on the concept of transfer learning is used to identify if a surgeon is experiencing any CWL. Secondly, a Convolutional Neural Network (CNN) uses this information to identify different degrees of CWL associated to each surgical task. The suggested multimodal approach considers adjacent signals from electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS) and eye pupil diameter. The concatenation of signals allows complex correlations in terms of time (temporal) and channel location (spatial). Data collection was performed by a Multi-sensing AI Environment for Surgical Task Role Optimisation platform (MAESTRO) developed at the Hamlyn Centre, Imperial College London. To compare the performance of the proposed methodology, a number of state-of-art machine learning techniques have been implemented. The tests show that the proposed model has a precision of 93


page 2

page 5


Aggregating Long-Term Context for Learning Surgical Workflows

Analyzing surgical workflow is crucial for computers to understand surge...

Surgical task-space optimisation of the CYCLOPS robotic system

The CYCLOPS is a cable-driven parallel mechanism used for minimally inva...

Real-time Surgical Tools Recognition in Total Knee Arthroplasty Using Deep Neural Networks

Total knee arthroplasty (TKA) is a commonly performed surgical procedure...

Learning Invariant Representation of Tasks for Robust Surgical State Estimation

Surgical state estimators in robot-assisted surgery (RAS) - especially t...

TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition

To enable context-aware computer assistance in the operating room of the...

Why Deep Surgical Models Fail?: Revisiting Surgical Action Triplet Recognition through the Lens of Robustness

Surgical action triplet recognition provides a better understanding of t...

Merging Deep Learning with Expert Knowledge for Seizure Onset Zone localization from rs-fMRI in Pediatric Pharmaco Resistant Epilepsy

Surgical disconnection of Seizure Onset Zones (SOZs) at an early age is ...

Please sign up or login with your details

Forgot password? Click here to reset