Multitask Learning of Temporal Connectionism in Convolutional Networks using a Joint Distribution Loss Function to Simultaneously Identify Tools and Phase in Surgical Videos

05/20/2019
by   Shanka Subhra Mondal, et al.
10

Surgical workflow analysis is of importance for understanding onset and persistence of surgical phases and individual tool usage across surgery and in each phase. It is beneficial for clinical quality control and to hospital administrators for understanding surgery planning. Video acquired during surgery typically can be leveraged for this task. Currently, a combination of convolutional neural network (CNN) and recurrent neural networks (RNN) are popularly used for video analysis in general, not only being restricted to surgical videos. In this paper, we propose a multi-task learning framework using CNN followed by a bi-directional long short term memory (Bi-LSTM) to learn to encapsulate both forward and backward temporal dependencies. Further, the joint distribution indicating set of tools associated with a phase is used as an additional loss during learning to correct for their co-occurrence in any predictions. Experimental evaluation is performed using the Cholec80 dataset. We report a mean average precision (mAP) score of 0.99 and 0.86 for tool and phase identification respectively which are higher compared to prior-art in the field.

READ FULL TEXT

page 2

page 8

page 9

research
07/13/2019

Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis

Surgical tool presence detection and surgical phase recognition are two ...
research
10/04/2017

Monitoring tool usage in cataract surgery videos using boosted convolutional and recurrent neural networks

With an estimated 19 million operations performed annually, cataract sur...
research
06/06/2018

Real-time Surgical Tools Recognition in Total Knee Arthroplasty Using Deep Neural Networks

Total knee arthroplasty (TKA) is a commonly performed surgical procedure...
research
05/22/2019

LapTool-Net: A Contextual Detector of Surgical Tools in Laparoscopic Videos Based on Recurrent Convolutional Neural Networks

We propose a new multilabel classifier, called LapTool-Net to detect the...
research
10/26/2020

A Joint Convolutional and Spatial Quad-Directional LSTM Network for Phase Unwrapping

Phase unwrapping is a classical ill-posed problem which aims to recover ...
research
02/24/2021

Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedures

Purpose: Automatic segmentation and classification of surgical activity ...
research
05/22/2018

A Recurrent Convolutional Neural Network Approach for Sensorless Force Estimation in Robotic Surgery

Providing force feedback as relevant information in current Robot-Assist...

Please sign up or login with your details

Forgot password? Click here to reset