Real-time image-based instrument classification for laparoscopic surgery

by   Sebastian Bodenstedt, et al.

During laparoscopic surgery, context-aware assistance systems aim to alleviate some of the difficulties the surgeon faces. To ensure that the right information is provided at the right time, the current phase of the intervention has to be known. Real-time locating and classification the surgical tools currently in use are key components of both an activity-based phase recognition and assistance generation. In this paper, we present an image-based approach that detects and classifies tools during laparoscopic interventions in real-time. First, potential instrument bounding boxes are detected using a pixel-wise random forest segmentation. Each of these bounding boxes is then classified using a cascade of random forest. For this, multiple features, such as histograms over hue and saturation, gradients and SURF feature, are extracted from each detected bounding box. We evaluated our approach on five different videos from two different types of procedures. We distinguished between the four most common classes of instruments (LigaSure, atraumatic grasper, aspirator, clip applier) and background. Our method succesfully located up to 86 respectively. On manually provided bounding boxes, we achieve a instrument type recognition rate of up to 58 to 49 To our knowledge, this is the first approach that allows an image-based classification of surgical tools in a laparoscopic setting in real-time.


page 3

page 8


1st Place Solution for the UVO Challenge on Image-based Open-World Segmentation 2021

We describe our two-stage instance segmentation framework we use to comp...

Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge

The ability to automatically detect and track surgical instruments in en...

Multi-label Classification of Surgical Tools with Convolutional Neural Networks

Automatic tool detection from surgical imagery has a multitude of useful...

Approximate Query Matching for Image Retrieval

Traditional image recognition involves identifying the key object in a p...

Fast and Regularized Reconstruction of Building Façades from Street-View Images using Binary Integer Programming

Regularized arrangement of primitives on building façades to aligned loc...

Please sign up or login with your details

Forgot password? Click here to reset