Leveraging Vision and Kinematics Data to Improve Realism of Biomechanic Soft-tissue Simulation for Robotic Surgery

03/14/2020
by   Jie Ying Wu, et al.
4

Purpose Surgical simulations play an increasingly important role in surgeon education and developing algorithms that enable robots to perform surgical subtasks. To model anatomy, Finite Element Method (FEM) simulations have been held as the gold standard for calculating accurate soft-tissue deformation. Unfortunately, their accuracy is highly dependent on the simulation parameters, which can be difficult to obtain. Methods In this work, we investigate how live data acquired during any robotic endoscopic surgical procedure may be used to correct for inaccurate FEM simulation results. Since FEMs are calculated from initial parameters and cannot directly incorporate observations, we propose to add a correction factor that accounts for the discrepancy between simulation and observations. We train a network to predict this correction factor. Results To evaluate our method, we use an open-source da Vinci Surgical System to probe a soft-tissue phantom and replay the interaction in simulation. We train the network to correct for the difference between the predicted mesh position and the measured point cloud. This results in 15-30 the mean distance, demonstrating the effectiveness of our approach across a large range of simulation parameters. Conclusion We show a first step towards a framework that synergistically combines the benefits of model-based simulation and real-time observations. It corrects discrepancies between simulation and the scene that results from inaccurate modeling parameters. This can provide a more accurate simulation environment for surgeons and better data with which to train algorithms.

READ FULL TEXT

page 5

page 6

research
10/26/2020

A 2D Surgical Simulation Framework for Tool-Tissue Interaction

The control and task automation of robotic surgical system is very chall...
research
01/01/2020

Simulation of Skin Stretching around the Forehead Wrinkles in Rhytidectomy

Objective: Skin stretching around the forehead wrinkles is an important ...
research
09/20/2023

Real-to-Sim Deformable Object Manipulation: Optimizing Physics Models with Residual Mappings for Robotic Surgery

Accurate deformable object manipulation (DOM) is essential for achieving...
research
09/09/2021

PhysGNN: A Physics-Driven Graph Neural Network Based Model for Predicting Soft Tissue Deformation in Image-Guided Neurosurgery

Correctly capturing intraoperative brain shift in image-guided neurosurg...
research
12/13/2018

Towards Fast Biomechanical Modeling of Soft Tissue Using Neural Networks

To date, the simulation of organ deformations for applications like ther...
research
03/12/2019

Suite of Meshless Algorithms for Accurate Computation of Soft Tissue Deformation for Surgical Simulation

The ability to predict patient-specific soft tissue deformations is key ...
research
11/07/2018

SurReal: enhancing Surgical simulation Realism using style transfer

Surgical simulation is an increasingly important element of surgical edu...

Please sign up or login with your details

Forgot password? Click here to reset