Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery

12/13/2020
by   Michael Zhang, et al.
6

Open, or non-laparoscopic surgery, represents the vast majority of all operating room procedures, but few tools exist to objectively evaluate these techniques at scale. Current efforts involve human expert-based visual assessment. We leverage advances in computer vision to introduce an automated approach to video analysis of surgical execution. A state-of-the-art convolutional neural network architecture for object detection was used to detect operating hands in open surgery videos. Automated assessment was expanded by combining model predictions with a fast object tracker to enable surgeon-specific hand tracking. To train our model, we used publicly available videos of open surgery from YouTube and annotated these with spatial bounding boxes of operating hands. Our model's spatial detections of operating hands significantly outperforms the detections achieved using pre-existing hand-detection datasets, and allow for insights into intra-operative movement patterns and economy of motion.

READ FULL TEXT

page 2

page 3

page 7

page 8

research
01/12/2021

Temporally Guided Articulated Hand Pose Tracking in Surgical Videos

Articulated hand pose tracking is an underexplored problem that carries ...
research
12/04/2018

Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos

Purpose: Real-time surgical tool tracking is a core component of the fut...
research
02/24/2018

Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks

Five billion people in the world lack access to quality surgical care. S...
research
10/12/2020

Towards human performance on automatic motion tracking of infant spontaneous movements

Assessment of spontaneous movements can predict the long-term developmen...
research
07/22/2022

Open video data sharing in developmental and behavioural science

Video recording is a widely used method for documenting infant and child...
research
08/22/2022

Automated Temporal Segmentation of Orofacial Assessment Videos

Computer vision techniques can help automate or partially automate clini...
research
09/13/2022

Computer vision based vehicle tracking as a complementary and scalable approach to RFID tagging

Logging of incoming/outgoing vehicles serves as a piece of critical info...

Please sign up or login with your details

Forgot password? Click here to reset