Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning

03/30/2020
by   Hannes Hase, et al.
11

In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task. Our method is trained and evaluated on an in-house collected data-set of 34 volunteers and when compared to pure RL and supervised learning (SL) techniques, it performs substantially better, which highlights the suitability of RL navigation for US-guided procedures. When testing our proposed model, we obtained a 82.91% chance of navigating correctly to the sacrum from 165 different starting positions on 5 different unseen simulated environments.

READ FULL TEXT

page 2

page 5

page 6

page 7

research
05/18/2018

Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrations

This work presents a learning-based approach for target driven map-less ...
research
08/08/2019

Vision-based Navigation Using Deep Reinforcement Learning

Deep reinforcement learning (RL) has been successfully applied to a vari...
research
06/06/2023

Reinforcement Learning-Based Control of CrazyFlie 2.X Quadrotor

The objective of the project is to explore synergies between classical c...
research
05/10/2022

VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation

Ultrasound (US) is one of the most common medical imaging modalities sin...
research
10/10/2022

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Deep reinforcement learning (RL) has brought many successes for autonomo...
research
12/09/2019

Reinforcement Learning based Visual Navigation with Information-Theoretic Regularization

We present a target-driven navigation approach for improving the cross-t...
research
11/12/2022

Deep Reinforcement Learning with Vector Quantized Encoding

Human decision-making often involves combining similar states into categ...

Please sign up or login with your details

Forgot password? Click here to reset