CholecTriplet2021: A benchmark challenge for surgical action triplet recognition

04/10/2022
by   Chinedu Innocent Nwoye, et al.
32

Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of <instrument, verb, target> combination delivers comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms by competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2 to 38.1 the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.

READ FULL TEXT

page 2

page 4

page 12

page 16

page 17

page 18

07/10/2020

Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets

Recognition of surgical activity is an essential component to develop co...
11/15/2017

Analyzing Before Solving: Which Parameters Influence Low-Level Surgical Activity Recognition

Automatic low-level surgical activity recognition is today well-known te...
04/11/2022

Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets

In addition to generating data and annotations, devising sensible data s...
09/30/2021

Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark

PURPOSE: Surgical workflow and skill analysis are key technologies for t...
09/07/2021

Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos

Out of all existing frameworks for surgical workflow analysis in endosco...
03/16/2022

Know your sensORs x2013 A Modality Study For Surgical Action Classification

The surgical operating room (OR) presents many opportunities for automat...
02/24/2021

Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedures

Purpose: Automatic segmentation and classification of surgical activity ...

Code Repositories

cholect45

Laparoscopic video dataset for surgical action triplet recognition


view repo

rendezvous

A transformer-inspired neural network for surgical action triplet recognition from laparoscopic videos.


view repo