Runtime Detection of Executional Errors in Robot-Assisted Surgery

03/01/2022
by   Zongyu Li, et al.
1

Despite significant developments in the design of surgical robots and automated techniques for objective evaluation of surgical skills, there are still challenges in ensuring safety in robot-assisted minimally-invasive surgery (RMIS). This paper presents a runtime monitoring system for the detection of executional errors during surgical tasks through the analysis of kinematic data. The proposed system incorporates dual Siamese neural networks and knowledge of surgical context, including surgical tasks and gestures, their distributional similarities, and common error modes, to learn the differences between normal and erroneous surgical trajectories from small training datasets. We evaluate the performance of the error detection using Siamese networks compared to single CNN and LSTM networks trained with different levels of contextual knowledge and training data, using the dry-lab demonstrations of the Suturing and Needle Passing tasks from the JIGSAWS dataset. Our results show that gesture specific task nonspecific Siamese networks obtain micro F1 scores of 0.94 (Siamese-CNN) and 0.95 (Siamese-LSTM), and perform better than single CNN (0.86) and LSTM (0.87) networks. These Siamese networks also outperform gesture nonspecific task specific Siamese-CNN and Siamese-LSTM models for Suturing and Needle Passing.

READ FULL TEXT

page 3

page 6

research
06/22/2021

Analysis of Executional and Procedural Errors in Dry-lab Robotic Surgery Experiments

Background We aim to develop a method for automated detection of potenti...
research
05/07/2020

Real-Time Context-aware Detection of Unsafe Events in Robot-Assisted Surgery

Cyber-physical systems for robotic surgery have enabled minimally invasi...
research
06/28/2023

Evaluating the Task Generalization of Temporal Convolutional Networks for Surgical Gesture and Motion Recognition using Kinematic Data

Fine-grained activity recognition enables explainable analysis of proced...
research
02/28/2023

Towards Surgical Context Inference and Translation to Gestures

Manual labeling of gestures in robot-assisted surgery is labor intensive...
research
01/28/2019

Context-aware Monitoring in Robotic Surgery

Robotic-assisted minimally invasive surgery (MIS) has enabled procedures...
research
09/24/2020

daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery

This paper presents a technique to concurrently and jointly predict the ...
research
08/24/2023

Robotic Scene Segmentation with Memory Network for Runtime Surgical Context Inference

Surgical context inference has recently garnered significant attention i...

Please sign up or login with your details

Forgot password? Click here to reset