A 2D Surgical Simulation Framework for Tool-Tissue Interaction

10/26/2020 ∙ by Yunhai Han, et al. ∙ University of California, San Diego 0

The control and task automation of robotic surgical system is very challenging, especially in soft tissue manipulation, due to the unpredictable deformations. Thus, an accurate simulator of soft tissues with the ability of interacting with robot manipulators is necessary. In this work, we propose a novel 2D simulation framework for tool-tissue interaction. This framework continuously tracks the motion of manipulator and simulates the tissue deformation in presence of collision detection. The deformation energy can be computed for the control and planning task.



There are no comments yet.


page 1

page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Recently, the expectations placed on surgical robotics have risen sharply. However, currently, surgeons are still required to manipulate the central control unit. In order to reduce surgeon fatigue, recent research has been extended to develop new control algorithms for surgical task automation [3]. The task can be divided into two parts:

  • Prediction of tissue deformation after a certain operation (e.g. input control commands).

  • Generation of operations based on the current and target deformation.

This work focuses on the first part, that is to simulate the 2D tissue deformation under the operation of a robotics manipulator.

The rest of the paper is organized as follows: Section II presents the detail of the framework; Section III shows the simulation results; Section IV gives the conclusion.

Ii Methodology

The simulation framework can be divided into four modules:

  • Mesh generation using 2D tissue images

  • Position-based dynamics methods for tissue simulation

  • Collision detection method for tool-tissue interaction

  • Implicit Euler energy computation

Ii-a Mesh Generation

Images only describe the geometric properties (e.g. shape) of the tissues, which are incomplete for any physical-based simulations. Hence, it is required to build the triangle/volumetric representations for the simulation of 2D/3D objects from the tissue images. In this work, a 2D surgical environment is generated using a standard tessellation algorithm (Delaunay Triangulator), as shown Fig. 1.

Fig. 1: A demonstration of mesh generation. The left image is the tissue image and the right image is the generated triangle meshes.

Ii-B PBD Simulation

In the recent decades, a position-based dynamics (PBD) method has earned increasing attention and has been shown to provide real-time performance and can be implemented efficiently [1], compared to traditional FEM, such as SOFA framework [2].

Ii-B1 Simulation Process

The deformed object is defined as a set of particles and constraints. The simulation process is described in Algorithm 1.

prediction step
while iter SolverIterations do
       for constraint  do
             Compute constraint solving step
       end for
end while
update position
update velocity
Algorithm 1 Simulation Process

In Algorithm 1, includes the gravity force and the driving force exerted by the manipulator when the collision is detected. However, in most cases, the driving force is unknown. In this work, instead of applying the driving force on the particles, we directly update their positions. You can image that there is an "invisible" force that leads to the position update in Equ 3.

Ii-B2 The Gauss-Seidel Method

In Algorithm 1, the position correction in each iteration can be computed through Gauss-Siedel Method [1]. In this work, the type of constraints are selected as follow:

  • Distance constraint: The distance constraint between each set of connected particles ( and ) can be satisfied by introducing:


    where is the initial distance indicated by rest spring length.

  • Area conservation: The area of a triangle, represented by three particles and , can be kept constant by introducing :


    where is the initial area of the triangle.

Ii-C Collision Detection for Tool-Tissue Interaction

Using flexible-collision-library111The usage documents can be found on github: python-fcl., the collision point is defined as the particle on the mesh when the distance () between it and the manipulator is smaller than a fixed threshold . The position update of particle is:


where, is the circle radius and is the velocity of the approaching manipulator.

Ii-D Implicit Euler Energy Computation

The definition of implicit Euler energy [1] contains both the inertial and potential energy, i.e.,:


where, and are the particles positions before and after constraint solving step in Algorithm 1, is the simulation time step and is the internal potential energy during constraint solving step, which is the sum of the following two parts : spring elastic energy and area conservation energy.

We make extensive use of the compliance form of elasticity as in [1]. Thus, the area conservation energy is defined by a quadratic potential energy in terms of the constraint function as,


where , where , and represents that the number of springs and number of triangle areas respectively, and is a block diagonal spring stiffness matrix.

Iii Experiments

In simulation, the boundary conditions are set as the upper and bottom horizontal lines shown in the right image in Fig 1. The velocity of the particle that crosses the horizontal lines are set as zero. The other simulation parameters (using metric system) are shown in Table I. Two experiments have been done for different tasks.

Num r
0.0001 0.15 1 -9.8 0.01 30 0.25 0.025
TABLE I: Simulation parameters

a. The tool is approaching the bottom tissue and tracking of energy are shown in Fig. 2.

Fig. 2: The energy tracking when the tool is approaching the bottom tissue. The vertical green dash lines are corresponded to the upper figure simulation steps.

b. We also track the energy variation of the tissues when the manipulator is inserted from two different angles which are aimed at the same target goal. As shown from Fig. 3, it can be used to find optimal control and planning policies via energy based cost definition.

Fig. 3:

A demonstration of energy estimation for two different insertion angles. The top-left and top-right figures show the final tissue deformation when the manipulator reaches the target goal (yellow circle), respectively. The bottom-left and bottom-right figures show the implicit Euler energy in the simulation process, respectively. It is obviously to see the energy variation according to insertion angle.

Iv Conclusion

Our simulation framework can successfully simulate the 2D tissue deformations under the manipulation of the robotic manipulator. The energy computation is suitable for the control and planning applications. Our system can also be easily extended to 3D surgical environments by considering 3D mass-spring network and tetrahedral volume constraints.


  • [1] J. Bender, M. Müller, and M. Macklin (2017) Position-based simulation methods in computer graphics. In EUROGRAPHICS 2017 Tutorials, External Links: Document Cited by: §II-B2, §II-B, §II-D, §II-D.
  • [2] F. Faure and C. Duriez (2012-06) SOFA: A Multi-Model Framework for Interactive Physical Simulation. Studies in Mechanobiology, Tissue Engineering and Biomaterials, Vol. 11, pp. 283–321. Cited by: §II-B.
  • [3] M. Yip and N. Das (2017-07) Robot autonomy for surgery. pp. . Cited by: §I.