Gesture Recognition in Robotic Surgery: a Review

01/29/2021
by   Beatrice van Amsterdam, et al.
13

Objective: Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions. Methods: An article search was performed on 5 bibliographic databases with the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling. Results: A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches. Conclusion: The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies. Significance: This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field.

READ FULL TEXT

page 1

page 3

page 5

page 8

page 9

page 10

page 11

research
12/03/2022

Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgery

Surgical activity recognition and prediction can help provide important ...
research
11/03/2020

Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery

Automatic surgical gesture recognition is fundamentally important to ena...
research
06/28/2023

Evaluating the Task Generalization of Temporal Convolutional Networks for Surgical Gesture and Motion Recognition using Kinematic Data

Fine-grained activity recognition enables explainable analysis of proced...
research
05/02/2021

Surgical Gesture Recognition Based on Bidirectional Multi-Layer Independently RNN with Explainable Spatial Feature Extraction

Minimally invasive surgery mainly consists of a series of sub-tasks, whi...
research
07/25/2019

Weakly Supervised Recognition of Surgical Gestures

Kinematic trajectories recorded from surgical robots contain information...
research
02/20/2020

Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search

Automatic surgical gesture recognition is fundamental for improving inte...
research
09/29/2022

Bounded Future MS-TCN++ for surgical gesture recognition

In recent times there is a growing development of video based applicatio...

Please sign up or login with your details

Forgot password? Click here to reset