Learning robotic ultrasound scanning using probabilistic temporal ranking

by   Michael Burke, et al.

This paper addresses a common class of problems where a robot learns to perform a discovery task based on example solutions, or human demonstrations. For example consider the problem of ultrasound scanning, where the demonstration requires that an expert adaptively searches for a satisfactory view of internal organs, vessels or tissue and potential anomalies while maintaining optimal contact between the probe and surface tissue. Such problems are currently solved by inferring notional rewards that, when optimised for, result in a plan that mimics demonstrations. A pivotal assumption, that plans with higher reward should be exponentially more likely, leads to the de facto approach for reward inference in robotics. While this approach of maximum entropy inverse reinforcement learning leads to a general and elegant formulation, it struggles to cope with frequently encountered sub-optimal demonstrations. In this paper, we propose an alternative approach to cope with the class of problems where sub-optimal demonstrations occur frequently. We hypothesise that, in tasks which require discovery, successive states of any demonstration are progressively more likely to be associated with a higher reward. We formalise this temporal ranking approach and show that it improves upon maximum-entropy approaches to perform reward inference for autonomous ultrasound scanning, a novel application of learning from demonstration in medical imaging.


page 1

page 4

page 5

page 6

page 7

page 8


Learning Ultrasound Scanning Skills from Human Demonstrations

Recently, the robotic ultrasound system has become an emerging topic owi...

Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations

Medical ultrasound has become a routine examination approach nowadays an...

Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward

Reinforcement learning often suffer from the sparse reward issue in real...

Enhanced Meta Reinforcement Learning using Demonstrations in Sparse Reward Environments

Meta reinforcement learning (Meta-RL) is an approach wherein the experie...

Identifying Visible Tissue in Intraoperative Ultrasound Images during Brain Surgery: A Method and Application

Intraoperative ultrasound scanning is a demanding visuotactile task. It ...

Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms

We provide new perspectives and inference algorithms for Maximum Entropy...

Pedagogical Demonstrations and Pragmatic Learning in Artificial Tutor-Learner Interactions

When demonstrating a task, human tutors pedagogically modify their behav...

Please sign up or login with your details

Forgot password? Click here to reset