Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge

by   Aneeq Zia, et al.

The ability to automatically detect and track surgical instruments in endoscopic videos can enable transformational interventions. Assessing surgical performance and efficiency, identifying skilled tool use and choreography, and planning operational and logistical aspects of OR resources are just a few of the applications that could benefit. Unfortunately, obtaining the annotations needed to train machine learning models to identify and localize surgical tools is a difficult task. Annotating bounding boxes frame-by-frame is tedious and time-consuming, yet large amounts of data with a wide variety of surgical tools and surgeries must be captured for robust training. Moreover, ongoing annotator training is needed to stay up to date with surgical instrument innovation. In robotic-assisted surgery, however, potentially informative data like timestamps of instrument installation and removal can be programmatically harvested. The ability to rely on tool installation data alone would significantly reduce the workload to train robust tool-tracking models. With this motivation in mind we invited the surgical data science community to participate in the challenge, SurgToolLoc 2022. The goal was to leverage tool presence data as weak labels for machine learning models trained to detect tools and localize them in video frames with bounding boxes. We present the results of this challenge along with many of the team's efforts. We conclude by discussing these results in the broader context of machine learning and surgical data science. The training data used for this challenge consisting of 24,695 video clips with tool presence labels is also being released publicly and can be accessed at


page 3

page 5

page 8

page 10

page 15

page 16

page 26

page 30


Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Surgical tool localization is an essential task for the automatic analys...

Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge

Timely and effective feedback within surgical training plays a critical ...

Real-time image-based instrument classification for laparoscopic surgery

During laparoscopic surgery, context-aware assistance systems aim to all...

Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision

In this work, we explore whether it is possible to learn representations...

Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos

Purpose: Real-time surgical tool tracking is a core component of the fut...

Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge

Surgical data science is revolutionizing minimally invasive surgery by e...

Analysis of Neural Fragility: Bounding the Norm of a Rank-One Perturbation Matrix

Over 15 million epilepsy patients worldwide do not respond to drugs and ...

Please sign up or login with your details

Forgot password? Click here to reset