Design and integration of a parallel, soft robotic end-effector for extracorporeal ultrasound

06/11/2019 ∙ by Lukas Lindenroth, et al. ∙ King's College London 0

In this work we address limitations in state-of-the-art ultrasound robots by designing and integrating the first soft robotic system for ultrasound imaging. It makes use of the inherent qualities of soft robotics technologies to establish a safe, adaptable interaction between ultrasound probe and patient. We acquire clinical data to establish the movement ranges and force levels required in prenatal foetal ultrasound imaging and design our system accordingly. The end-effector's stiffness characteristics allow for it to reach the desired workspace while maintaining a stable contact between ultrasound probe and patient under the determined loads. The system exhibits a high degree of safety due to its inherent compliance in the transversal direction. We verify the mechanical characteristics of the end-effector, derive and validate a kinetostatic model and demonstrate the robot's controllability with and without external loading. The imaging capabilities of the robot are shown in a tele-operated setting on a foetal phantom. The design exhibits the desired stiffness characteristics with a high stiffness along the ultrasound transducer and a high compliance in lateral direction. Twist is constrained using a braided mesh reinforcement. The model can accurately predict the end-effector pose with a mean error of about 6 controller is, with an average position error of 0.39mm able to track a target pose efficiently without and with externally applied loads. Finally, the images acquired with the system are of equally good quality compared to a manual sonographer scan.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 5

page 6

page 9

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

It is commonly accepted that sonographers are exposed to an increased risk in repetitive strain injury [1, 2, 3]. A representative study amongst diagnostic medical sonographers and vascular technologists indicates that a significant majority of sonographers experience pain while performing ultrasound scans [4]. This suggests a high demand to improve ergonomics and offload sonographers during clinical scan procedures. Recent investigations show that besides diagnostic sonography, there is an increased demand for intraoperative transthoracic [5, 6] and transoesophegal [7] ultrasound imaging, particularly for cardiac and lung procedures. Sonographers performing intraoperative ultrasound in for example cardiac catheterization procedures have therefore presumably an increased risk of radiation exposure [8].

Fig. 1: Soft robotic end-effector (SEE) performing ultrasound scan on abdominal prenatal phantom

Automating diagnostic and intraoperative ultrasound procedures through robot-guidance or -assistance can help address the aforementioned problems and lay the groundwork for more intelligent image acquisition. Robotic ultrasound guidance has found particular application in procedures involving steering orthopaedic [9] or minimally-invasive surgical tools [10] and biopsy needles [11]. Various robotic hardware solutions have been proposed. Researchers have adopted robotic platforms originally aimed at collaborative scenarios in industrial settings, such as Universal Robot’s UR-series [12, 13] or the KUKA LWR [9] and LBR iiwa [14, 15]. A commercial robotic manipulator has been released (LBR Med, KUKA AG, Augsburg, Germany) which is suitable for use in clinical environments due to its conformity with medical device safety (ISO 60601) and medical software regulations (ISO 62304). Current research suggests that such robots can be applied in diagnostics to autonomously perform aorta measurements [16], in combination with previously acquired MRI scans to autonomously find standard view-planes [17] and in intraoperative procedures to autonomously track surgical tools [18], amongst others. Whilst such robotic platforms allow for great flexibility through a large workspace and high manipulability, the use of large-scale robotic manipulators can pose various disadvantages for clinical integration. Diagnostic ultrasound scans are divided into their respective body area of interest. For an individual procedure such as a lower abdominal ultrasound scan, a robotic system is therefore only required to achieve a workspace to cover a fraction of the human body. This yields that common robotic manipulators are oversized for such applications, which unnecessarily poses risks to patient safety. Despite high degrees of electrical safety, a mechanical system with a high mass can potentially be more dangerous.

To address this issue, researchers developed customized solutions which are tailored to the application-specific requirements of diagnostic and interventional sonography. Researchers [19, 20, 21] have proposed a mechanism which achieves a high degree of probe manipulability and safety. The robot actuation has been moved to the base of the system, thus minimizing its size and weight. Other systems have been developed which separate the probe positioning into two stages: approximate probe placement and finer view-plane adjustments. The first can be achieved by a passive positioning mechanism, which is operated by a clinician, while the latter is obtained with an active end-effector. Researchers from Waseda University first proposed this concept and corresponding design in [22], in which the end-effector is driven through a parallel mechanism. Similarly, a consortium of researchers have developed a system with active end-effector with the aim of remote tele-diagnosis [23, 24, 25]. The system has since been trialled for remote scans [26] and translated to a commercial product (MELODY, AdEchoTech, Naveil, France). Despite the scanning being performed remotely, the design of the system suggests, however, that the assisting operator is still required to apply the necessary force to maintain a stable contact.

Maintaining stable mechanical coupling between ultrasound probe and patient tissue is of paramount importance for ensuring a high-quality image. Approaches to achieve this involve controlling the contact force directly or establishing an elastic contact between the position-controlled device and the patient. While the first has been researched extensively [27] and can be commonly found in various forms of industrial applications, the latter has found more attention in recent years due to an increased demand in cost effective force control and -limiting solutions for human robot collaboration tasks [28, 29]. Series-elastic actuators have been developed to provide passive compliance in actuated robotic joints [30]. While providing a degree of compliance, this has the disadvantage that a collision or undesired contact in a direction other than the joint axis cannot be compensated for. We have trialled safety clutches for the use in ultrasound robots which exhibit compliant behaviour once disengaged through an excess force [31, 32]. This, however, renders the system uncontrollable and requires reengaging the clutch mechanism for further operation. In this work, we make use of an elastic soft robotic system, which is aimed at overcoming aforementioned limitations. Soft robotics technologies have opened up new design paradigms for robotic systems through the use of elastic and deformable materials and structures [33, 34]. Soft robotics systems are commonly designed to interact with or conform to environmental contacts. This allows soft robotic manipulators to exhibit highly dexterous manoeuvrability in for example surgical [35, 36, 37] or search and rescue operations [38]. In these scenarios, however, soft robots are not applied to tasks which require significant loadbearing capabilities, predominantly due to their low stiffness. To bridge the trade-off between manoeuvrability and stiffness, research has been driven towards systems with variable stiffness capabilities. A comprehensive overview of stiffening technologies is given in [39]. For applications in which softness is desired, high loadings are demanded and stiffening mechanisms are not suitable, soft robotic systems tend to be combined with external constraints to ensure structural integrity. This is commonly found in exoskeleton research and rehabilitation robotics. Examples include full body, soft exosuits [40], lower limb exoskeletons [41] and hand exoskeletons for post-stroke rehabilitation [42, 43].

In this work, we derive the first soft robotic end-effector which is capable of safely acquiring standard views in extracorporeal diagnostic foetal ultrasound (US). We select foetal US as an initial application due to its high demands to robot safety. We evaluate the performance of our system with respect to derived specifications and show that the proposed system is capable of acquiring a set of standard view-plane required for the assessment of the foetus. The robot utilizes linear soft fluidic actuators (SFAs) which are arranged in parallel around the ultrasound probe to provide high axial loadbearing capabilities and high lateral compliance, thus enabling adaptability and safety in the patient interaction. The individual contributions of this study are:

  • Clinical investigation to determine workspace and force requirements for view-plane adjustments in foetal diagnostic ultrasound imaging.

  • Design and verification of a soft robotic end-effector which satisfies the derived clinical requirements in workspace and force. It employs robust linear soft fluidic actuators, for which a novel injection-based fabrication is derived, and undesired twist is prevented through a mesh constraint.

  • Definition and validation of a lumped stiffness model to describe the motion of the soft robotic end-effector in the absence and presence of external loading.

The controllability and imaging capabilities of the integrated system are validated in position control and US phantom experiments respectively.

The paper is structured in the following way. In Section II-A the system requirements are determined, and the robot design is introduced. Based on the design of the system, Section II-B derives a kinetostatic model. Methodologies for the actuation and control of the system are presented in Section II-C. In Section II-D the mechanical properties of the system and its workspace are evaluated. Results are presented in section III. The proposed model is validated and the position controller performance, as well as the imaging capabilities of the system, are assessed.

Ii Methods

Prenatal foetal ultrasound is a routine diagnostic procedure for pregnant women to determine birth defects and abnormalities in the foetus. Common checks include measuring the foetus’ biparietal diameter (BPD), its head and abdominal circumferences (HC and AC) as well as its femur length (FL) [44].

In this work we focus on obtaining HC, AC and FL standard view-planes. We establish the clinical requirements to the contact force and movement range of the ultrasound probe for bespoke application and derive a suitable design for a soft robotic end-effector (SEE).

Fig. 2: Proposed design of the soft robotic end-effector
Fig. 3: Braided nylon mesh uncrimped (a) and crimped (b)

Ii-a Design

Ii-A1 Clinical data acquisition and processing

Pregnant women between 18 to 24 weeks of gestation underwent research ultrasound scans at St Thomas’ Hospital (Study title: Intelligent Fetal Imaging and Diagnosis (iFIND)-2: Further Ultrasound and MR Imaging, Study reference: 14/LO/1806). Trained sonographers performed the foetal ultrasound scan using a standard ultrasound probe (X6-1, Philips, Amsterdam, Netherlands) which is connected to an ultrasound scanner (EPIQ7, Philips, Amsterdam, Netherlands). The probe was placed in a holder as detailed in [45]

. This holder incorporated an electromagnetic (EM) tracking sensor (Aurora, NDI, Ontario, Canada) and six axis force-torque sensor (Nano 17, ATI, Apex, USA), which allowed measurements of the position and orientation of the probe, and the force applied at the probe face to be measured throughout the scan. The recorded tracking and force data of six patients were analysed by extracting time ranges during which standard fetal anomaly views were imaged. These included HC, AC and FL views. Each time range consisted of the few seconds when the sonographer had placed the probe in the correct anatomical region and was adjusting the probe to find the ideal view. For each view the tracking data were analysed to find the range of positions and orientations in the three axes separately. The X and Y axes show movement in the horizontal plane of the scanning bed (left to right on the patient, and foot to head, respectively), and the Z axis shows vertical movement. Orientation ranges are given in probe coordinates, with yaw showing axial rotation, pitch showing elevational tilting out of the image plane, and roll showing in-plane lateral tilting. Forces were analysed by dividing the measured force vector into normal and tangential components applied to the surface. The local surface angle was determined at each measurement by fitting an ellipsoidal shape to the tracking data of the scan. The 95th percentile of the forces measured within a time range gives an indication of the maximum force that must be applied by the probe.

Ii-A2 Mechanism requirements and synthesis

Following the results of the clinical data analysis, it is found that the soft robotic end-effector must satisfy the following requirements

  • Be able to withstand axial contact forces of up to N and maximum transversal contact forces of N

  • Maximum translation in a range of mm

  • Maximum tilt in a range of

Fig. 4: Free body diagram of SEE model definition

To maintain a high degree of safety when interacting with the device, the SEE should furthermore comprise of a low transversal stiffness. This allows both the operating clinician and patient to manually displace the probe in case of discomfort.

A soft robotic design based on soft fluidic actuators (SFAs), which have previously been presented in [46], is proposed. It is comprised of two rigid platforms which serve as base and transducer holder respectively. The platforms are connected through a set of three soft fluidic actuators which are arranged in a parallel fashion at intervals. To allow for sufficient space for the ultrasound transducer cable, the actuators are tilted at an angle of 15. An overview of the design is shown in Figure 2.

A common problem in such a proposed soft robotic system is the low stiffness along its twist axis. To improve the stability of the system against twist deformations, a nylon fibre mesh is attached to base and transducer platforms, which acts as a mechanical constraint between the two. To reduce unwanted buckling behaviour, crimps can be added to the mesh by deforming and heat-treating it. Examples of uncrimped and crimped meshes are shown in Figure 3. Thus, axial rotation of the ultrasound transducer is not considered in this study, as it could be added by simply applying a rotating mechanism to the base of the SEE, which would function as a stiff rotational axis in conjunction with the mesh constraint.

Ii-B Kinetostatic modelling

To determine the ultrasound probe pose under internal fluid volume variation and external loading a kinetostatic model is derived according to [47]. A free body diagram of the model is shown in Figure 4. In the following, a vector denoted as

represents a 6 degree of freedom wrench in an arbitrary frame

such that with forces

and moments

. Similarly, denotes a reaction wrench in the local SFA frame, which is of the same form as . Vectors noted as indicate infinitesimally small displacements in frame f of the form with translations and rotations .

Let be a vector of forces and moments applied to the tip of the ultrasound transducer. Under static equilibrium conditions, the following holds for a single actuator

(1)

Where is the wrench caused by the elastic deformation of the SFA and is the reaction wrench caused by the constrained hydraulic chamber. Both are expressed in the tip frame of the system. The tip wrenches and can be expressed relative to their local frames by

(2)

Where is a vector of local reaction forces and moments caused by the SFA deformation and is the uniaxial reaction force of the volumetric constraint in the actuator. The matrices and are defined by

(3)

is the rotation matrix of the current tip deflection. Matrix is the wrench transformation matrix relating the local SFA frame to the tip frame by

(4)

Where is the spatial rotation of the respective frame and is the cross-product matrix with the translation vector . is for a single SFA a 6x1 vector containing the third column of .

Considering the elastic behaviour of the SFA, its reaction force caused by an infinitesimally small, local displacement can be written as

(5)

Where the SFA stiffness is defined as a Timoshenko beam element with

describes the length of the SFA, it’s cross-sectional area, its Young’s modulus, the area moment of inertia, its torsion constant and the torsion constant. The Timoshenko coefficient is defined as

with the Timoshenko coefficient . An overview of the SFA constants is given in Table I.

Constant Value
mm
kPa
E
TABLE I: Model parameters

For a given SFA volume, the kinematic relationship between an infinitesimal small volume change of the SFA and the displacement of the ultrasound tip frame is given by

(6)

The kinematic motion of the tip frame caused by the SFA deflection can be defined as

(7)

Substituting Equation 7 into 5 yields

(8)

Applying Equations 3 and 8, the static equilibrium condition in Equation 1 can be written as

(9)

Equation 9 can be combined with the imposed kinematic constraint defined by Equation 6 to a linear equation system of the form

(10)

The deflection of the ultrasound transducer tip and internal reaction of the system can consequently be found through matrix inversion

(11)

The formulation can be expanded to a number of SFAs by considering a lumped stiffness in the probe tip frame. As the actuators are aligned in a parallel configuration, it can be defined by

(12)

The matrix is adopted by appending the respective columns of the wrench transformation matrix of actuator to such that

(13)

The kinematic constraint relationship then becomes

(14)

Where is an vector of SFA volume changes.

To account for changes in matrices and for a given motion, the model is solved numerically by dividing the applied external wrench and induced volume vectors into small increments . After each iteration, is updated according to the previous tip pose.

Fig. 5: Actuation unit with syringe pumps and controller system

For the given number of three SFAs, the update rule for the numerical solution is defined by

(15)

For iteration step .

Fig. 6: Overview of mould components (i)-(vi) and drawing of final SFA (vii)

Ii-C Actuation and control

The SEE is actuated by inflating respective SFAs with a working fluid. As shown in our previous work [48], we utilize custom hydraulic syringe pumps which are driven by stepper motors (Nema 17, Pololu Corporation, Las Vegas, USA) to induce volume changes in the SFAs. The pumps are controlled with a microcontroller (Teensy 3.5, PJRC, Sherwood, USA) which communicates via a serial interface with a PC running ROS (Intel Core I7-7700HQ, XPS15 9560, Dell, Texas, USA). The PC generates demand velocities or positions for the microcontroller and solves the previously-defined kinetostatic model to determine the system Jacobian for a given pose. Furthermore, the laptop handles interfaces with peripherals such as a joystick (Spacemouse Compact, 3dconnexion, Monaco) and for closed-loop position control an electromagnetic tracking (EM) system (Aurora, NDI, Ontario, Canada).

Ii-C1 Soft fluidic actuators

The linear soft fluidic actuators utilized in this work have been conceptualized in previous work [46]. They are comprised of a silicone rubber body (Dragonskin 10-NV, SmoothOn Inc, Pennsylvania, USA) and stiffer silicone rubber endcaps (SmoothSil 945, SmoothOn Inc, Pennsylvania, USA). A helical constraint is inserted into the silicone to counteract radial expansion of the actuator upon inflation. This, in combination with the stiff endcaps, allows for the actuators to maintain its form and only expand in the direction of actuation. The moulding process of creating SFAs has been significantly improved from our previous work. For the radial constraint an extension spring (Figure 6(v)) is used. The liquid silicone rubber is injected through an inlet (Figure 6(ii)) using a syringe instead of being poured into the mould. This has the significant advantage for the user to be able to pre-assemble the mould without having to manually wind the constraint helix, as it has been commonly done in soft fluidic actuators [49]. In combination with the injection of the silicone this could reduce variations in the fabrication process. A drawing of a finished actuators is shown in Figure 6(vii). The combination of radial constraint and stiff endcaps allows for the actuators to be driven efficiently with a volumetric input without exhibiting nonlinear behaviour due to bulging.

Ii-C2 Closed-loop position control

The Jacobian-based closed-loop position controller is designed as a linear PI-controller with feedback being provided through EM tracking. A high-level trajectory generator continuously updates the demand position for the position controller, which generates in return demand volumes for the three syringe pumps. The gain constants and have been verified experimentally and are defined as and respectively. The kinetostatic model feeds back the transposed Jacobian to the controller, which is used to generate demand volume changes. The target points are generated at 2Hz while both the position controller and the kinetostatic model are updated at 30Hz. The low-level step generation for driving the syringe pumps is achieved with an update rate of 6kHz.

Ii-D Experimental validation

SFA characterization

Using the fabricated SFAs in an open-loop configuration requires the volume-extension relation to be predictable for any given point in time. This is determined experimentally using a linear rail setup. The position of the tip of the actuator is equipped with a slider and tracked using a linear potentiometer. Contact friction between the linear bearings and rails is minimized using lubrication and friction forces are therefore neglected in the evaluation of the results. Volume and extension data are tracked and synchronized using ROS.

Fig. 7: Experimental setup for stiffness characterization
Fig. 8: SEE moving in contact with soft rubber patch
Stiffness characterization

The stiffness of the system is characterized with the setup shown in Figure 7. The SEE is mounted to a base plate and its tip is connected through a force-torque sensor (Gamma, ATI, Apex, USA) to a robot manipulator (UR3, Universal Robots, Odense, Denmark). To determine the stiffness of the SEE in a given direction, the manipulator moves the SEE in said direction and the resulting reaction force is measured. This is repeated 10 times for each configuration. The linearized relationship between reaction force and manipulator displacement corresponds to the stiffness of the SEE.

The mesh reinforcement’s effect on the axial twist stiffness is determined by twisting the SEE repeatedly by and measuring the z-axis moment. This is done for a configuration without mesh reinforcement, mesh reinforcement without crimps and mesh reinforcement with crimps.

The directional lateral stiffness is obtained by displacing the SEE tip radially in a defined direction over a distance of 10mm. This is repeated for four inflation levels (25%, 50%, 75% and 100% of the maximum SFA volume) and for directions between 0 and in increments around the z-axis. The axial stiffness which corresponds to each extension is determined by displacing the SEE tip in negative z-direction by 1.5mm for 25% and 50% inflation, and by 2.5mm for 75% and 100% extension.

Fig. 9: Sonographer performing SEE-assisted ultrasound scanning of a prenatal abdominal phantom (iii). The SEE (i) is attached to a passive arm (ii) and manually placed on the phantom. A joystick (iv) is used to manipulate the ultrasound probe under visual guidance of the acquired image (v)
Fig. 10: SFA pressure (a) and extension (b) under increasing working fluid volume
(a) Change in stiffness with direction
(b) Change in stiffness with extension
(c) Change in stiffness with bending
Fig. 11: Stiffness characteristics of the SEE
Workspace and repeatability

The workspace of the SEE is measured using an electromagnetic tracker (6DOF Reference, Aurora, NDI, Ontario, Canada) which is attached to the side of the SEE tip. The pose of the ultrasound probe tip is calculated with the known homogeneous transformation between tracker and tip. The SFA volumes are varied between 0% and 100% in 10% increments and the resulting static tip pose is determined with respect to its deflated state.

The repeatability in positioning the tip of the SEE is determined by repeatedly approaching defined SFA volume states and measuring the tip pose. A set of 6 states is defined and the resultant trajectory is executed 50 times.

Model validation

The derived model is validated by comparing the workspace and corresponding SFA volumes to the calculated tip pose of the SEE.

Indentation behaviour

To verify the effect a soft tissue-like contact has on the SEE, a soft mechanical phantom is created. The cylindrical phantom is moulded from a layer of Ecoflex Gel and a structural layer of Ecoflex 00-30 (SmoothOn Inc, Pennsylvania, USA).

The tip of the SEE is controlled to perform a line trajectory from its negative to positive x-axis limits at 60% inflation. The tip pose is monitored with a magnetic tracker and contact forces between SEE and phantom are measured using an ATI Gamma at the base of the phantom. The manipulator is used to test for different indentation depths from 0mm to 15mm in 5mm increments.

Controllability

The controllability of the SEE is verified with the closed-loop Jacobian-based control system described in Section II-C2. Target trajectories are defined as isosceles triangles with a base of 12.33mm and height of 10mm. For the tilted trajectory, the triangle is titled about one of its sides by 19. The trajectory is tested in a planar and tilted configuration and tracked 3 times each.

To determine the controllability under an external load, a stiff silicone rubber patch is created as shown in Figure 14. The patch is lubricated and positioned with its center at the tip of the SEE. To ensure contact with the patch, an initial axial force of 5N is generated by displacing the patch and running the position controller. This is repeated for planar and tilted configurations, where each trajectory is tracked 3 times.

Sonographer-guided teleoperation

The imaging capabilities for an ultrasound transducer guided by the SEE are verified using a prenatal abdominal phantom (SPACE FAN-ST, Kyoto Kagaku, Japan). The SEE is equipped with an abdominal ultrasound probe (X6-1, Philips, Amsterdam, Netherlands) which is connected to an ultrasound scanner (EPIQ7, Philips, Amsterdam, Netherlands). A passive positioning arm (Field Generator Mounting Arm, NDI, Ontario, Canada) is used to manually position the SEE in the region of interest on the phantom. The sonographer uses the provided ultrasound image feedback to steer the SEE with a connected joystick towards a desired view-plane. The target view-planes have been acquired manually using a handheld ultrasound probe. An overview of the experimental setup is shown in Figure 9.

Iii Results

Iii-a Clinical data

The results of the clinical data acquisition are presented in Table II. The mean and maximum values of ultrasound probe absolute displacement, tilt and applied contact force are shown for each subject. For subject 2 only HC and AC views were obtained. Displacements and tilts are shown with respect to the patient bed. The normal force is assumed to be acting only in negative probe direction and the tangential force shows the vector magnitude of the lateral forces in X and Y.

View-plane adjustments require a maximum translation of mm by mm by mm. Corresponding maximum tilts of pitch and roll are in ranges of and . In this study the axial rotation of the probe, yaw, is ignored. The maximum occurring normal and tangential forces are 14.1N and 8.9N respectively.

Subject Views Max. transl. [mm] Max. tilt [] Max. force [N]
x y z yaw pitch roll normal tangential
1 HC, AC, FL 8.5 7.9 5.7 6.2 5 7.7 13.8 4.1
2 HC, AC 13.5 6.4 7.5 10.8 5.9 12.9 14.1 6.1
3 HC, AC, FL 9.7 12.4 8 10.3 9.8 7.8 13.3 4.9
4 HC, AC, FL 11.9 8.4 7 14.8 4.7 9.4 10.1 6.9
5 HC, AC, FL 13.1 20 10 14.8 5 7.1 4.4 3.6
6 HC, AC, FL 5.7 10 7.6 7 7.8 4.1 8.3 8.9
max 13.5 20 10 14.8 9.8 12.9 14.1 8.9
10.4 11 7.7 10.7 6.4 8.2 10.7 5.8
TABLE II: Clinical data

Iii-B SFA characterization

The results of the SFA characterization are shown in Figure 10. The hydraulic pressure under SFA inflation and the resulting extension are shown in Figure 10(a) and 10(b) respectively. As hysteresis effects are ignored in this study, the curves depict the average between inflation and deflation paths.

The volume-extension curve of the SFA can be separated into two regions, a nonlinear (0ml to 1.25ml) and a linear region (1.25ml to 5ml). In the linear region, the relationship can be can be approximated with a first order polynomial as . As the proportion of the nonlinear region compared to the overall extension of the SFA is small, it is ignored for the following investigations. SFAs are therefore assumed to be pre-extended with a volume of 1.25ml.

None Uncrimped Crimped
[Nmm/] 45.94 84.68 137.37
[Nmm/] 0.70 4.30 2.73
TABLE III: Twist stiffness
Fig. 12: Workspace of the SEE in position (a-b) and orientation (c-d). The coloured markers indicate states in which the two SFAs other than the one indicated are of equal volume

Iii-C Twist stiffness

The results of the twist stiffness characterization for each mesh configuration are shown in Table III. The application of a nylon mesh helps to significantly stiffen the torsional axis of the system by 184%. A crimped mesh can further improve the torsional stiffness to 299% of its original value.

Iii-D Directional stiffness

The results of the lateral stiffness characterization under inflation of the SEE are shown in Figure 10(a)

) in polar coordinates. The radius indicates the magnitude of the stiffness in the given direction. For the given inflation levels of 25% to 100% the corresponding standard deviations in lateral stiffness are 0.21N/mm, 0.20 N/mm, 0.18 N/mm and 0.30 N/mm. The averaged lateral stiffness of the SEE as a function of its z-axis extension is shown in Figure

10(b)). For varying extension, the axial stiffness of the SEE is on average 10.12 times greater the lateral stiffness. Under bending, which is presented in Figure 10(c)) the ratio decreases to 8.66 times.

Iii-E Workspace

The workspace of the SEE in position and orientation is shown in Figure 12. The figures show the tip pose acquired by the EM tracker for any given SFA configuration. Coloured markers indicate configurations in which one SFA is inflated while the others remain at the same level. The resulting planes spun by the markers therefore indicated the movement planes of each actuator. It is shown that a maximum combined lateral deflection of 19.01mm can be reached along the principal plane of . The maximum extension of the SEE of 22.13mm is reached for a full inflation of all SFAs. The maximum tilt of the SEE is reached along the principal plane of with 14.02. A maximum axial torsion of 1.03 occurs. Compared to the tilt ranges in X and Y the twist is significantly lower and will therefore be ignored in the following investigations.

Fig. 13: Workspace generated with model in position (a) and orientation (b). The colour indicates the normalized Euclidean error in the given state with respect to the maximum error of the model
Pose
0.09 0.05
TABLE IV: Repeatability

The results of the positioning repeatability evaluation are presented in Table IV. The table indicates the mean Euclidean errors in position and orientation with their respective standard deviations from the given pose for the 50 repetitions in each direction. The measured accuracy of the SEE is with mm in position and orientation high and lies below the rated accuracy of the EM tracker of 0.8mm or 0.7 RMS.

Iii-F Model validation

The results of the model validation are shown in Figure 13 and summarized in Table V

. The estimated workspace of the SEE generated with the kinetostatic model is shown in Figure

13. The colour of each marker indicates the Euclidean distance between the calculated point and the corresponding measured pose normalized to the maximum error in position and orientation respectively. Overall, the model validation shows with a mean Euclidean error of mm in position and in orientation, which are equivalent to 6.21% and 6.56% of the maximum lateral deflection and tilt respectively, good results in predicting the tip pose under SFA extension.

Displacement [mm] Tilt []
max max
-0.81 0.20 1.25 0.04 0.44 1.34
-0.55 0.47 1.60 0.05 0.86 2.26
-0.05 0.50 1.80 -0.13 0.36 1.36
1.18 0.29 2.37 0.92 0.47 2.47
TABLE V: Model validation
Fig. 14: Effect of axial loading on transversal motion
Fig. 15: Example of tracked trajectory under external loading. The normal force is represented with scale of 0.2mm/N

Iii-G Contact experiment

The motion constraint induced by an indentation contact is investigated. Figure 14 shows the constraint of the mean x-displacement and y-tilt for a given motion over 10 repetitions normalized to the maximum value. The lateral force applied by the SEE is measured with the force torque sensor. For both, the displacement and the tilt, the magnitude declines monotonically. Whereas the displacement reaches a minimum at 27.84%, the tilt remains less affected by the lateral force with a minimum of 55.35%. Linearizing the trends yield a decrease of 14.09 for the displacement and only 8.56 for the tilt.

Flat - Unloaded Tilted - Loaded Flat - Loaded Tilted - Loaded
max max max max
[mm] 0.20 0.19 0.98 0.19 0.19 0.82 0.24 0.23 1.10 0.27 0.28 1.50
[mm] 0.25 0.20 1.03 0.27 0.20 0.88 0.33 0.22 1.08 0.32 0.25 1.07
[mm] 0.11 0.10 0.52 0.13 0.07 0.35 0.13 0.10 0.56 0.17 0.10 0.65
[mm] 0.34 0.29 1.51 0.36 0.28 1.25 0.42 0.33 1.64 0.45 0.39 1.95
TABLE VI: Position control results

Iii-H Position control

An example of a tracked trajectory with external loading is shown in Figure 15. The position controller tracks the desired position accurately with marginally larger tracking error around the corners of the triangular path. The quantitative results of the controller evaluation for the three executions are presented in Table VI for both the unloaded and loaded trajectories. The results indicate a higher mean error for the z-direction regardless of the configuration, which is also observable in the visualization above.

Fig. 16: Ultrasound images acquired by sonographer (a-c) and SEE (d-f) for HC (a,d), AC (b,e) and FL (c,f) measurements

Iii-I Teleoperation and image-acquisition

The images obtained through manual ultrasound probe placement and steering with the SEE are presented in Figure 16. Anatomical structures of the foetus phantom are clearly visible throughout all images with minor shadowing on the left side of the FL standard view-plane, outside of the region of interest. In both cases, the regions of interest are centered in the image. Moreover, the contrast in the robot-acquired images is similar to the one in the manually-obtained images.

Iv Discussion

In this work we developed a soft robotic ultrasound imaging system to improve sonographers’ ergonomics, allow for tele-operation of the scanning and provide a platform for advanced imaging approaches. The device is designed in form of an end-effector which is positioned in the area of interest and actively steered towards the desired view-plane. Due to its inherent compliance, the SEE is able to maintain contact while exhibiting sufficient axial stiffness to ensure mechanical coupling for the ultrasound image acquisition.

The system shows with its high axial and low lateral stiffness good applicability to foetal ultrasound scanning. Despite the quick decline of stiffness with axial extension, the SEE is with axial stiffness at full extension still capable to apply sufficiently high forces to the patient without significant deformation (about 1.03mm at maximum axial load during intervention of 14.1N). The lower lateral stiffness allows for the system to be adaptable to the contact surface and be moved away in case of discomfort in the patient. We have shown that the integration of a braided nylon mesh, which has previously only been used to avoid ballooning in SFAs, can significantly improve the twist stiffness of the SEE to up to three times in comparison to the mesh-free system. This is crucial for spatial loadbearing tasks and greatly improves the usability of the system.

The coupling between position and orientation is an obvious limitation in the usability of the design. It can be seen, however, that the mechanical properties of the surface contact greatly affect the coupling behaviour. We have shown that an indenting contact reduces the lateral motion of the ultrasound probe significantly more than the tilt. It can easily be seen that a very stiff coupling in combination with the minimal contact friction caused by the application of ultrasound gel greatly reduces the tilt capabilities of the system while allowing for lateral sliding. It can therefore be concluded that in practice the coupling can be reduced by varying the axial pressure applied to the patient. This is supported by the findings of the tele-operated image acquisition in Section III-I and will be investigated further in future research.

A maximum lateral motion and tilt range is achieved when no axial extension is present. To ensure a more homogeneous workspace, an axial translation degree of freedom could be added to the system without sacrificing the adaptable performance of the system.

The validation of the proposed model indicates high accuracy for most of the workspace with about 6% error in deflection and 7% in tilt. The model deviates more along the boundaries of the workspace, which could be caused by the larger deflection of the SFAs and resultant nonlinearieties in the material properties. It can be computed efficiently, and we have shown its suitability for the integration in open-loop velocity and closed-loop position control systems. The position-controlled system can track target positions accurately, both in free space and under external loading.

V Conclusion

The SEE design proposed in this work is a first proof of concept for utilizing soft robotics technologies in medical ultrasound applications. We have shown that under certain conditions the SEE satisfies the requirements imposed by the clinical application. The derived kinetostatic model mimics adequately the behaviour of the physical robot and the integrated system is capable of tracking target trajectories accurately and obtaining high-quality ultrasound images of a prenatal ultrasound phantom.

References

  • [1] E. Seto and L. Biclar, “Ambidextrous sonographic scanning to reduce sonographer repetitive strain injury,” Journal of Diagnostic Medical Sonography, vol. 24, no. 3, pp. 127–135, 2008.
  • [2] D. Janga and O. Akinfenwa, “Work-related repetitive strain injuries amongst practitioners of obstetric and gynaecological ultrasound worldwide,” Archives of Gynecology and Obstetrics, vol. 286, no. 2, pp. 353–356, 2012.
  • [3] G. Harrison and A. Harris, “Work-related musculoskeletal disorders in ultrasound: Can you reduce risk?,” Ultrasound, vol. 23, no. 4, pp. 224–230, 2015.
  • [4] K. Evans, S. Roll, and J. Baker, “Work-related musculoskeletal disorders (WRMSD) among registered diagnostic medical sonographers and vascular technologists: A representative sample,” Journal of Diagnostic Medical Sonography, vol. 25, no. 6, pp. 287–299, 2009.
  • [5] I. Ben-Dor, M. R. Kramer, A. Raccah, Z. Iakobishvilli, D. Shitrit, G. Sahar, and D. Hasdai, “Echocardiography versus right-sided heart catheterization among lung transplantation candidates,” Annals of Thoracic Surgery, vol. 81, no. 3, pp. 1056–1060, 2006.
  • [6] K. Hori, T. Matsuura, T. Mori, and K. Nishikawa, “Usefulness and growing need for intraoperative transthoracic echocardiography: A case series,” BMC Anesthesiology, vol. 15, no. 1, pp. 1–4, 2015.
  • [7] J. S. Shanewise, A. T. Cheung, S. Aronson, W. J. Stewart, R. L. Weiss, J. B. Mark, R. M. Savage, P. Sears-Rogan, J. P. Mathew, M. A. Quiñones, M. K. Cahalan, and J. S. Savino, “ASE/SCA Guidelines for Performing a Comprehensive Intraoperative Multiplane Transesophageal Echocardiography Examination: Recommendations of the American Society of Echocardiography Council for Intraoperative Echocardiography and the Society of Cardiovasc,” Journal of the American Society of Echocardiography, vol. 12, pp. 884–900, 10 1999.
  • [8] E. F. McIlwain, P. D. Coon, A. J. Einstein, C. K. Mitchell, G. W. Natello, R. A. Palma, M. M. Park, F. Ranallo, and M. L. Roberts, “Radiation safety for the cardiac sonographer: Recommendations of the radiation safety writing group for the council on cardiovascular sonography of the American society of echocardiography,” Journal of the American Society of Echocardiography, vol. 27, no. 8, pp. 811–816, 2014.
  • [9] P. J. Gonçalves, P. M. Torres, F. Santos, R. António, N. Catarino, and J. M. Martins, “A Vision System for Robotic Ultrasound Guided Orthopaedic Surgery,” Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 77, no. 2, pp. 327–339, 2014.
  • [10] M. Antico, F. Sasazawa, L. Wu, A. Jaiprakash, J. Roberts, R. Crawford, A. K. Pandey, and D. Fontanarosa, “Ultrasound guidance in minimally invasive robotic procedures,” Medical Image Analysis, 1 2019.
  • [11] M. Z. Mahmoud, M. Aslam, M. Alsaadi, M. A. Fagiri, and B. Alonazi, “Evolution of Robot-assisted ultrasound-guided breast biopsy systems,” Journal of Radiation Research and Applied Sciences, vol. 11, pp. 89–97, 1 2018.
  • [12] K. Mathiassen, J. E. Fjellin, K. Glette, P. K. Hol, and O. J. Elle, “An Ultrasound Robotic System Using the Commercial Robot UR5,” Frontiers in Robotics and AI, vol. 3, no. February, pp. 1–16, 2016.
  • [13] H. T. Şen, A. Cheng, K. Ding, E. Boctor, J. Wong, I. Iordachita, and P. Kazanzides, “Cooperative Control with Ultrasound Guidance for Radiation Therapy,” Frontiers in Robotics and AI, vol. 3, no. August, pp. 1–12, 2016.
  • [14] R. Kojcev, B. Fuerst, O. Zettinig, J. Fotouhi, S. C. Lee, B. Frisch, R. Taylor, E. Sinibaldi, and N. Navab, “Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop,” International Journal of Computer Assisted Radiology and Surgery, vol. 11, no. 6, pp. 1173–1181, 2016.
  • [15] O. Zettinig, B. Fuerst, R. Kojcev, M. Esposito, M. Salehi, W. Wein, J. Rackerseder, E. Sinibaldi, B. Frisch, and N. Navab, “Toward real-time 3D ultrasound registration-based visual servoing for interventional navigation,” Proceedings - IEEE International Conference on Robotics and Automation, vol. 2016-June, pp. 945–950, 2016.
  • [16] S. Virga, O. Zettinig, M. Esposito, K. Pfister, B. Frisch, T. Neff, N. Navab, and C. Hennersperger, “Automatic force-compliant robotic Ultrasound screening of abdominal aortic aneurysms,” IEEE International Conference on Intelligent Robots and Systems, vol. 2016-Novem, pp. 508–513, 2016.
  • [17] C. Hennersperger, B. Fuerst, S. Virga, O. Zettinig, B. Frisch, T. Neff, and N. Navab, “Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study,” IEEE Transactions on Medical Imaging, vol. 36, no. 2, pp. 538–548, 2017.
  • [18] S. E. Salcudean, O. Mohareri, P. Abolmaesumi, T. K. Adebar, and M. Ramezani, “Automatic Localization of the da Vinci Surgical Instrument Tips in 3-D Transrectal Ultrasound,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 9, pp. 2663–2672, 2013.
  • [19] S. Salcudean, G. Bell, and S. Bachmann, “Robot-assisted diagnostic ultrasound–design and feasibility experiments,” Proc. Int. Conf. in Medical Image Computing and Computer-Assisted Intervention (), pp. 1062–1072, 1999.
  • [20] S. E. Salcudean, W. H. Zhu, P. Abolmaesumi, S. Bachmann, and P. D. Lawrence, “A robot system for medical ultrasound Electromechanical desing,” Robotics Research, the 9th International Symposium of Robotics Research (ISRR’99), J.M. Hollerbach and D.E. Koditschek (Eds.), pp. 195–202, 1999.
  • [21] W.-h. Z. M. R. S. S. P. D. Purang Abolmaesumi, Septimiu E Salcudean, “Image-Guided Control of a Robot for,” IEEE Transactions on Robotics and Automation, vol. 18, no. 1, pp. 11–23, 2002.
  • [22] R. Nakadate, H. Uda, H. Hirano, J. Solis, A. Takanishi, E. Minagawa, M. Sugawara, and K. Niki, “Development of a robotic carotid blood measurement WTA-1RII: Mechanical improvement of gravity compensation mechanism and optimal link position of the parallel manipulator based on GA,” IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM, pp. 717–722, 2009.
  • [23] a. Gourdon, P. Poignet, G. Poisson, P. Vieyres, and P. Marche, “A new robotic mechanism for medical application,” 1999 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Cat. No.99TH8399), pp. 33–38, 1999.
  • [24] P. Arbeille, G. Poisson, P. Vieyres, J. Ayoub, M. Porcher, and J. L. Boulay, “Echographic examination in isolated sites controlled from an expert center using a 2-D echograph guided by a teleoperated robotic arm,” Ultrasound in Medicine and Biology, vol. 29, no. 7, pp. 993–1000, 2003.
  • [25] P. Vieyres, G. Poisson, F. Courrèges, O. Mérigeaux, and P. Arbeille, “The TERESA project: From space research to ground tele-echography,” Industrial Robot, vol. 30, no. 1, pp. 77–82, 2003.
  • [26] P. Arbeille, J. Ruiz, P. Herve, M. Chevillot, G. Poisson, and F. Perrotin, “Fetal tele-echography using a robotic arm and a satellite link,” Ultrasound in Obstetrics and Gynecology, vol. 26, no. 3, pp. 221–226, 2005.
  • [27] B. Siciliano and L. Villani, Robot Force Control. Springer US, 1999.
  • [28] W. McMahan, V. Chitrakaran, M. Csencsits, D. Dawson, I. D. Walker, B. A. Jones, M. Pritts, D. Dienno, M. Grissom, and C. D. Rahn, “Field trials and testing of the OctArm continuum manipulator,” Proceedings - IEEE International Conference on Robotics and Automation, vol. 2006, no. May, pp. 2336–2341, 2006.
  • [29] A. Albu-Schaffer, O. Eiberger, M. Grebenstein, S. Haddadin, C. Ott, T. Wimbock, S. Wolf, and G. Hirzinger, “Soft robotics,” IEEE Robotics & Automation Magazine, vol. 15, no. 3, pp. 20–30, 2008.
  • [30] G. Pratt and M. Williamson, “Series elastic actuators,” in Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots, vol. 1, pp. 399–406, IEEE Comput. Soc. Press.
  • [31] S. Wang, R. J. Housden, Y. Noh, A. Singh, L. Lindenroth, H. Liu, K. Althoefer, J. Hajnal, D. Singh, and K. Rhode, “Analysis of a Customized Clutch Joint Designed for the Safety Management of an Ultrasound Robot,” Applied Sciences, vol. 9, no. 9, p. 1900, 2019.
  • [32] S. Wang, J. Housden, Y. Noh, A. Singh, J. Back, L. Lindenroth, H. Liu, J. Hajnal, K. Althoefer, D. Singh, and others, “Design and Implementation of a Bespoke Robotic Manipulator for Extra-corporeal Ultrasound,” JoVE (Journal of Visualized Experiments), no. 143, p. e58811, 2019.
  • [33] C. Laschi, B. Mazzolai, and M. Cianchetti, “Soft robotics: Technologies and systems pushing the boundaries of robot abilities,” Science Robotics, vol. 1, no. 1, pp. 1–12, 2016.
  • [34] P. Polygerinos, N. Correll, S. A. Morin, B. Mosadegh, C. D. Onal, K. Petersen, M. Cianchetti, M. T. Tolley, and R. F. Shepherd, “Soft Robotics: Review of Fluid-Driven Intrinsically Soft Devices; Manufacturing, Sensing, Control, and Applications in Human-Robot Interaction,” Advanced Engineering Materials, vol. 19, p. 1700016, 12 2017.
  • [35] M. Cianchetti, T. Ranzani, G. Gerboni, T. Nanayakkara, K. Althoefer, P. Dasgupta, and A. Menciassi, “Soft Robotics Technologies to Address Shortcomings in Today’s Minimally Invasive Surgery: The STIFF-FLOP Approach,” Soft Robotics, vol. 1, no. 2, pp. 122–131, 2014.
  • [36] A. D. Marchese, K. Komorowski, C. D. Onal, and D. Rus, “Design and Control of a Soft and Continuously Deformable 2D Robotic Manipulation System,” Proceedings of IEEE International Conference on Robotics and Automation. 2014, pp. 2189–2196, 5 2014.
  • [37] J. Burgner-Kahrs, D. C. Rucker, and H. Choset, “Continuum Robots for Medical Applications: A Survey,” 2015.
  • [38] E. W. Hawkes, L. H. Blumenschein, J. D. Greer, and A. M. Okamura, “A soft robot that navigates its environment through growth,” Science Robotics, vol. 2, no. 8, 2017.
  • [39] M. Manti, V. Cacucciolo, and M. Cianchetti, “Stiffening in soft robotics: A review of the state of the art,” IEEE Robotics and Automation Magazine, vol. 23, no. 3, pp. 93–106, 2016.
  • [40] M. Wehner, B. Quinlivan, P. M. Aubin, E. Martinez-villalpando, L. Stirling, K. Holt, R. Wood, and C. Walsh, “A Lightweight Soft Exosuit for Gait Assistance,” 2013 IEEE International Conference on Robotics and Automation, pp. 3362–3369, 2013.
  • [41] N. Costa and D. G. Caldwell, “Control of a biomimetic ”soft-actuated” 10DoF lower body exoskeleton,” in Proceedings of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006, BioRob 2006, vol. 2006, pp. 495–501, IEEE, 2006.
  • [42] A. Chiri and N. Vitiello, “Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation,” Mechatronics, IEEE/ …, vol. 17, no. 5, pp. 884–894, 2012.
  • [43] A. Stilli, A. Cremoni, M. Bianchi, A. Ridolfi, F. Gerii, F. Vannetti, H. A. Wurdemann, B. Allotta, and K. Althoefer, “AirExGlove — A novel pneumatic exoskeleton glove for adaptive hand rehabilitation in post-stroke patients,” in 2018 IEEE International Conference on Soft Robotics (RoboSoft), pp. 579–584, IEEE, 4 2018.
  • [44] L. J. Salomon, Z. Alfirevic, V. Berghella, C. Bilardo, E. Hernandez-Andrade, S. L. Johnsen, K. Kalache, K. Y. Leung, G. Malinger, H. Munoz, F. Prefumo, A. Toi, and W. Lee, “Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan,” Ultrasound in Obstetrics and Gynecology, vol. 37, no. 1, pp. 116–126, 2011.
  • [45] Y. Noh, R. J. Housden, A. Gomez, C. Knight, F. Garcia, H. Liu, R. Razavi, K. Rhode, and K. Althoefer, “An ergonomic handheld ultrasound probe providing contact forces and pose information,” Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, vol. 2015-Novem, pp. 5773–5776, 2015.
  • [46] L. Lindenroth, A. Soor, J. Hutchinson, A. Shafi, J. Back, K. Rhode, and H. Liu, “Design of a soft, parallel end-effector applied to robot-guided ultrasound interventions,” in IEEE International Conference on Intelligent Robots and Systems, vol. 2017-Septe, pp. 3716–3721, 2017.
  • [47] A. Klimchik, A. Pashkevich, and D. Chablat, “MSA-technique for stiffness modeling of manipulators with complex and hybrid structures,” IFAC-PapersOnLine, vol. 51, no. 22, pp. 37–43, 2018.
  • [48] L. Lindenroth, J. Back, A. Schoisengeier, Y. Noh, H. Wurdemann, K. Althoefer, and H. Liu, “Design and implementation of a hydraulically-actuated soft robotics manipulator,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016.
  • [49] K. Suzumori, S. Iikura, and H. Tanaka, “Development of flexible microactuator and its applications to robotic mechanisms,” in Proceedings. 1991 IEEE International Conference on Robotics and Automation, pp. 1622–1627, IEEE Comput. Soc. Press, 2002.