LapGym – An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

02/19/2023
by   Paul Maria Scheikl, et al.
0

Recent advances in reinforcement learning (RL) have increased the promise of introducing cognitive assistance and automation to robot-assisted laparoscopic surgery (RALS). However, progress in algorithms and methods depends on the availability of standardized learning environments that represent skills relevant to RALS. We present LapGym, a framework for building RL environments for RALS that models the challenges posed by surgical tasks, and sofa_env, a diverse suite of 12 environments. Motivated by surgical training, these environments are organized into 4 tracks: Spatial Reasoning, Deformable Object Manipulation Grasping, Dissection, and Thread Manipulation. Each environment is highly parametrizable for increasing difficulty, resulting in a high performance ceiling for new algorithms. We use Proximal Policy Optimization (PPO) to establish a baseline for model-free RL algorithms, investigating the effect of several environment parameters on task difficulty. Finally, we show that many environments and parameter configurations reflect well-known, open problems in RL research, allowing researchers to continue exploring these fundamental problems in a surgical context. We aim to provide a challenging, standard environment suite for further development of RL for RALS, ultimately helping to realize the full potential of cognitive surgical robotics. LapGym is publicly accessible through GitHub (https://github.com/ScheiklP/lap_gym).

READ FULL TEXT

page 3

page 9

page 10

page 11

page 12

page 13

page 14

page 21

research
03/05/2019

Open-Sourced Reinforcement Learning Environments for Surgical Robotics

Reinforcement Learning (RL) is a machine learning framework for artifici...
research
06/25/2021

Multi-Goal Reinforcement Learning environments for simulated Franka Emika Panda robot

This technical report presents panda-gym, a set Reinforcement Learning (...
research
06/16/2023

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Open-source reinforcement learning (RL) environments have played a cruci...
research
06/24/2020

The NetHack Learning Environment

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand wit...
research
05/03/2021

Robotic Surgery With Lean Reinforcement Learning

As surgical robots become more common, automating away some of the burde...
research
03/14/2021

Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems

Active network management (ANM) of electricity distribution networks inc...
research
01/01/2023

Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning

Surgical robot automation has attracted increasing research interest ove...

Please sign up or login with your details

Forgot password? Click here to reset