Patient-Specific Domain Adaptation for Fast Optical Flow Based on Teacher-Student Knowledge Transfer

by   Sontje Ihler, et al.

Fast motion feedback is crucial in computer-aided surgery (CAS) on moving tissue. Image-assistance in safety-critical vision applications requires a dense tracking of tissue motion. This can be done using optical flow (OF). Accurate motion predictions at high processing rates lead to higher patient safety. Current deep learning OF models show the common speed vs. accuracy trade-off. To achieve high accuracy at high processing rates, we propose patient-specific fine-tuning of a fast model. This minimizes the domain gap between training and application data, while reducing the target domain to the capability of the lower complex, fast model. We propose to obtain training sequences pre-operatively in the operation room. We handle missing ground truth, by employing teacher-student learning. Using flow estimations from teacher model FlowNet2 we specialize a fast student model FlowNet2S on the patient-specific domain. Evaluation is performed on sequences from the Hamlyn dataset. Our student model shows very good performance after fine-tuning. Tracking accuracy is comparable to the teacher model at a speed up of factor six. Fine-tuning can be performed within minutes, making it feasible for the operation room. Our method allows to use a real-time capable model that was previously not suited for this task. This method is laying the path for improved patient-specific motion estimation in CAS.


page 2

page 6

page 7

page 9


Semi-Supervised Learning of Optical Flow by Flow Supervisor

A training pipeline for optical flow CNNs consists of a pretraining stag...

Conditional Teacher-Student Learning

The teacher-student (T/S) learning has been shown to be effective for a ...

Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation

We present DistillFlow, a knowledge distillation approach to learning op...

Explicit Knowledge Transfer for Weakly-Supervised Code Generation

Large language models (LLMs) can acquire strong code-generation capabili...

Improving Fast Segmentation With Teacher-student Learning

Recently, segmentation neural networks have been significantly improved ...

Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation

Cardiac motion estimation plays a key role in MRI cardiac feature tracki...

Knowledge Flow: Improve Upon Your Teachers

A zoo of deep nets is available these days for almost any given task, an...

Please sign up or login with your details

Forgot password? Click here to reset