Open-Sourced Reinforcement Learning Environments for Surgical Robotics

by   Florian Richter, et al.

Reinforcement Learning (RL) is a machine learning framework for artificially intelligent systems to solve a variety of complex problems. Recent years has seen a surge of successes solving challenging games and smaller domain problems, including simple though non-specific robotic manipulation and grasping tasks. Rapid successes in RL have come in part due to the strong collaborative effort by the RL community to work on common, open-sourced environment simulators such as OpenAI's Gym that allow for expedited development and valid comparisons between different, state-of-art strategies. In this paper, we aim to bridge the RL and the surgical robotics communities by presenting the first open-sourced reinforcement learning environments for surgical robotics, called dVRL. Through the proposed RL environment, which are functionally equivalent to Gym, we show that it is easy to prototype and implement state-of-art RL algorithms on surgical robotics problems that aim to introduce autonomous robotic precision and accuracy to assisting, collaborative, or repetitive tasks during surgery. Learned policies are furthermore successfully transferable to a real robot. Finally, combining dVRL with the over 40+ international network of da Vinci Surgical Research Kits in active use at academic institutions, we see dVRL as enabling the broad surgical robotics community to fully leverage the newest strategies in reinforcement learning, and for reinforcement learning scientists with no knowledge of surgical robotics to test and develop new algorithms that can solve the real-world, high-impact challenges in autonomous surgery.


page 1

page 2

page 3

page 5

page 6

page 7


LapGym – An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

Recent advances in reinforcement learning (RL) have increased the promis...

Robotic Surgery With Lean Reinforcement Learning

As surgical robots become more common, automating away some of the burde...

SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

Autonomous surgical execution relieves tedious routines and surgeon's fa...

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

Success stories of applied machine learning can be traced back to the da...

Etat de l'art des systèmes robotisés en vue d'une application pour la chirurgie otologique

This article deals with a patents state-of-the-art linked to our robotic...

ARtonomous: Introducing Middle School Students to Reinforcement Learning Through Virtual Robotics

Typical educational robotics approaches rely on imperative programming f...

Decision-Theoretic Approaches for Robotic Environmental Monitoring – A Survey

Robotics has dramatically increased our ability to gather data about our...

Please sign up or login with your details

Forgot password? Click here to reset