Unsupervised Temporal Video Segmentation as an Auxiliary Task for Predicting the Remaining Surgery Duration

by   Dominik Rivoir, et al.

Estimating the remaining surgery duration (RSD) during surgical procedures can be useful for OR planning and anesthesia dose estimation. With the recent success of deep learning-based methods in computer vision, several neural network approaches have been proposed for fully automatic RSD prediction based solely on visual data from the endoscopic camera. We investigate whether RSD prediction can be improved using unsupervised temporal video segmentation as an auxiliary learning task. As opposed to previous work, which presented supervised surgical phase recognition as auxiliary task, we avoid the need for manual annotations by proposing a similar but unsupervised learning objective which clusters video sequences into temporally coherent segments. In multiple experimental setups, results obtained by learning the auxiliary task are incorporated into a deep RSD model through feature extraction, pretraining or regularization. Further, we propose a novel loss function for RSD training which attempts to counteract unfavorable characteristics of the RSD ground truth. Using our unsupervised method as an auxiliary task for RSD training, we outperform other self-supervised methods and are comparable to the supervised state-of-the-art. Combined with the novel RSD loss, we slightly outperform the supervised approach.


page 1

page 2

page 3

page 4


Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillation

Despite the outstanding success of self-supervised pretraining methods f...

Self-Supervised Feature Extraction for 3D Axon Segmentation

Existing learning-based methods to automatically trace axons in 3D brain...

CataNet: Predicting remaining cataract surgery duration

Cataract surgery is a sight saving surgery that is performed over 10 mil...

RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations

Objective: Accurate surgery duration estimation is necessary for optimal...

Prediction of laparoscopic procedure duration using unlabeled, multimodal sensor data

The course of surgical procedures is often unpredictable, making it diff...

Lateral Connections in Denoising Autoencoders Support Supervised Learning

We show how a deep denoising autoencoder with lateral connections can be...

Please sign up or login with your details

Forgot password? Click here to reset