Robotic Surgery With Lean Reinforcement Learning

by   Yotam Barnoy, et al.

As surgical robots become more common, automating away some of the burden of complex direct human operation becomes ever more feasible. Model-free reinforcement learning (RL) is a promising direction toward generalizable automated surgical performance, but progress has been slowed by the lack of efficient and realistic learning environments. In this paper, we describe adding reinforcement learning support to the da Vinci Skill Simulator, a training simulation used around the world to allow surgeons to learn and rehearse technical skills. We successfully teach an RL-based agent to perform sub-tasks in the simulator environment, using either image or state data. As far as we know, this is the first time an RL-based agent is taught from visual data in a surgical robotics environment. Additionally, we tackle the sample inefficiency of RL using a simple-to-implement system which we term hybrid-batch learning (HBL), effectively adding a second, long-term replay buffer to the Q-learning process. Additionally, this allows us to bootstrap learning from images from the data collected using the easier task of learning from state. We show that HBL decreases our learning times significantly.



There are no comments yet.


page 1

page 5


Open-Sourced Reinforcement Learning Environments for Surgical Robotics

Reinforcement Learning (RL) is a machine learning framework for artifici...

UbuntuWorld 1.0 LTS - A Platform for Automated Problem Solving & Troubleshooting in the Ubuntu OS

In this paper, we present UbuntuWorld 1.0 LTS - a platform for developin...

SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

Autonomous surgical execution relieves tedious routines and surgeon's fa...

Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research

The purpose of this technical report is two-fold. First of all, it intro...

A Scavenger Hunt for Service Robots

Creating robots that can perform general-purpose service tasks in a huma...

Towards a Common Implementation of Reinforcement Learning for Multiple Robotic Tasks

Mobile robots are increasingly being employed for performing complex tas...

Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations

Reinforcement Learning (RL) has achieved significant success in applicat...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.