FBG-Based Position Estimation of Highly Deformable Continuum Manipulators: Model-Dependent vs. Data-Driven Approaches

12/20/2018 ∙ by Shahriar Sefati, et al. ∙ Johns Hopkins University Applied Physics Laboratory Johns Hopkins University 0

Conventional shape sensing techniques using Fiber Bragg Grating (FBG) involve finding the curvature at discrete FBG active areas and integrating curvature over the length of the continuum dexterous manipulator (CDM) for tip position estimation (TPE). However, due to limited number of sensing locations and many geometrical assumptions, these methods are prone to large error propagation especially when the CDM undergoes large deflections. In this paper, we study the complications of using the conventional TPE methods that are dependent on sensor model and propose a new data-driven method that overcomes these challenges. The proposed method consists of a regression model that takes FBG wavelength raw data as input and directly estimates the CDM's tip position. This model is pre-operatively (off-line) trained on position information from optical trackers/cameras (as the ground truth) and it intra-operatively (on-line) estimates CDM tip position using only the FBG wavelength data. The method's performance is evaluated on a CDM developed for orthopedic applications, and the results are compared to conventional model-dependent methods during large deflection bendings. Mean absolute TPE error (and standard deviation) of 1.52 (0.67) mm and 0.11 (0.1) mm with maximum absolute errors of 3.63 mm and 0.62 mm for the conventional and the proposed data-driven techniques were obtained, respectively. These results demonstrate a significant out-performance of the proposed data-driven approach versus the conventional estimation technique.



There are no comments yet.


page 1

page 2

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Due to their flexibility, continuum dexterous manipulators (CDMs) can be used in surgical tasks where significant dexterity and maneuverability are necessary. Compared to rigid link robots, CDMs can adopt various shapes, therefore, enhance the reach of surgeons in confined spaces [1]. Though beneficial for complex surgical tasks, a CDM’s flexibility makes accurate shape sensing and TPE challenging. Further, in many applications CDMs (usually the tip) are guided to target locations inside the body, requiring accurate position sensing for safe navigation.

In recent years, optical fibers such as Fiber Bragg Gratings (FBG) have been the focus of attention for shape and tip sensing of biopsy needles (e.g.[2]) and continuum manipulators [3]. FBGs are flexible, small in size and can provide data at high frequencies without requiring a direct line of sight making them perfectly suitable for CDM position sensing, especially for surgical applications. The fibers are typically either combined with small elastic substrates to form a standalone sensor [4] or attached directly to the CDMs [6].

Fig. 1: Continuum manipulator developed for orthopedic applications as well as the integrated FBG sensor with triangular configuration. The sensor consists of three fibers, each with three active areas.

A variety of FBG sensor manufacturing techniques associated with the choice of substrate material, geometry, number of fibers and their attachments have been explored in the literature [4]-[12]. Liu et al. [4] used a triangular configuration (Fig. 2) with one fiber (three gratings) and two Nitinol (NiTi) substrates. They first found the curvature at discrete locations by calibrating the FBG wavelength data to constant curvatures and then performing TPE indirectly using this calibration. Roesthuis et al. [5] attached three fibers each with four active areas to a NiTi substrate and found curvature and bending plane by modeling the sensor assembly’s cross section. Xu et al. [6] proposed a helically wrapped sensor with three FBG fibers for simultaneous curvature/torsion/force measurements for concentric tube robots. Wei et al. [7] also suggested a helically wrapped sensor using only one fiber with multiple FBG nodes.

A common technique for FBG-based TPE is to find curvature at discrete locations, extrapolate curvature along the length of the sensor, compute slopes and reconstruct the shape from base to tip [3]. In a triangular configuration (Fig. 2), three FBG nodes at each cross sections are used to simultaneously find curvature, plane of bend, and the common strain at active sensing locations [2]. This method of TPE, however, relies on many geometrical assumptions (model-dependent) contributing to inaccuracies in estimations of curvature and plane of bend. Moreover, due to the limited number of active sensing locations and shape reconstruction from base to tip, the TPE will be prone to error, especially in large deflections.

To address the mentioned drawbacks in conventional model-dependent shape reconstruction and TPE techniques, we propose a new data-driven approach that uses only the sensory data to learn the necessary parameters for estimating the CDM’s tip position. The trained model does not rely on any assumption regarding the geometrical relations of the manufactured sensor and is therefore expected to reduce the error in CDM TPE especially when CDM undergoes large deflections. The contributions of this paper are as follows: 1) proposing a data-driven model-independent approach using only sensory data to estimate CDM tip position; 2) implementation and analysis of both the proposed and the conventional TPE methods on a large deflection CDM developed for orthopedic applications; 3) error analysis of the conventional model-dependent technique on large deflection CDMs; and 4) comparison of TPE error using different number of fibers (one, two, or three) in the data-driven approach.

The remainder of the paper is organized as follows. Section II reviews the conventional model-dependent approach and describes the proposed data-driven method for CDM TPE. Section III explains the CDM and FBG sensor used in this study, and describes the experiments conducted for evaluation of the proposed and conventional methods. Section V includes the TPE results and discussion on the obtained data from the experiments. In section VI the paper is concluded and future work is presented.

Ii Methods

As the CDM bends, the raw FBG wavelength data changes due to strain changes at the FBG nodes. Conventional shape reconstruction methods [3]-[5] aim at estimating curvature at active sensing locations based on the sensor’s geometrical model and then integrating over the sensor’s length to estimate the tip position. A few drawbacks of this method are 1) curvature is found at each cross section using only the local FBG nodes in that specific cross section 2) at minimum, three FBG nodes are required at each active cross section 3) several assumptions regarding the sensor design are made during the shape reconstruction step 4) extra calibration steps are required to find geometrical parameters. To remedy the above-mentioned shortcomings, we propose a data-driven approach where the correlation of all FBG nodes on the sensor is used simultaneously to predict the CDM tip position. In addition, the data-driven approach does not make any geometrical or structural assumptions regarding the sensor design.

Ii-a Conventional Model-dependent Method

Fig. 2: FBG sensor and its cross section in triangular configuration. Geometrical parameters used in conventional model-dependent approach are shown.

In model-dependent approaches, first the strain at each FBG node is found by (1) and then based on the sensor’s geometry, a system of nonlinear equations (2) is solved at each cross section with three FBG nodes (fibers a, b, and c placed in a triangular configuration (Fig. 2) to find curvature (), its angle (), and a common strain bias () at that cross section of the sensor:


where is the Bragg wavelength, is the strain constant for optical fiber and , are geometrical parameters and can be assumed known from design or be found by an extra calibration procedure [2]. Then assuming a relationship (typically linear) between curvature and arc length, and dividing the sensor length to sufficiently small segments, curvature () and its direction () can be extrapolated at each segment (3). Using the curvature at each segment ( for ) slope of each segment can be found using (4). By attaching an appropriate local coordinate frame to the beginning of each segment, shape can be reconstructed segment by segment to find the CDM tip position using (5):


Using (5) for , the tip position of the sensor () is found. For CDMs in which the FBG sensor is not placed on the center axis of the CDM, all s should then be shifted by (distance between sensor’s and CDM’s centers) to obtain the 3-D position of the center-line of the CDM.

Ii-B Data-driven Method

To improve the accuracy of TPE, we propose a data-driven method that does not rely on any assumptions regarding the design and manufacturing of the sensor or the number of FBG nodes at each cross section. In addition, the proposed method simultaneously uses data from all FBG nodes across the sensor length for TPE.

FBG is a suitable sensing modality during a surgery where there is no or limited direct line of sight. Moreover, it can stream data at high frequencies (e.g. KHz), which is ideal for real-time CDM tip position control. Other sensing modalities such as optical trackers or stereo cameras can stream the data in lower frequencies, however, can measure the CDM tip position directly with high accuracy. Therefore, a model can be trained on the FBG raw wavelength data as input and the true CDM tip position from another sensor (optical tracker or cameras) as output. The training phase can be carried out pre-operatively (off-line) and the trained model can be used to predict the CDM tip position intra-operatively using only the FBG raw wavelength data as input. A regression model is suitable for the proposed task, since given some independent variables as input, prediction of a dependant variable is desired.

For the problem at hand, we aim to find a regression model that can predict the CDM 3-D tip position (dependent variables) given a set of raw FBG wavelength data along the sensor (independent variables):


where is the 3-D position of the CDM tip,

is the vector containing the raw wavelength data of the

m FBG nodes on the sensor, is the vector of unknown parameters, and

is the regression model that predicts CDM 3-D tip position, given the wavelength information of the complete set of FBG nodes on the sensor at any given time. Depending on the degrees of freedom of the CDM, complexity of the environment, and the shapes that the CDM can obtain, different regression models could be used to capture the unknown parameters

. Linear regression is a common regression model that models the dependent variables as a linear combination of the unknown parameters. The CDM tip position sensing can be modeled as a least squares optimization problem:


where is the residual of the error for the observation, is a stack of N observations of the m FBG node data, is the stack of N true CDM tip position observation data from optical tracker or cameras, and is the matrix of unknown parameters. Using (7), the regression model is trained pre-operatively on N observations of the FBG and true CDM tip position (section III-B) to find the unknown parameters using the generalized inverse:


The trained model can then be used intra-operatively to predict CDM tip position values given only the current FBG wavelength data:


where is the predicted CDM tip position, given the current wavelength data for all FBG nodes on the sensor ().

Using the proposed regression model, the uncertainties regarding the shape sensor manufacturing such as FBG locations will be captured in the unknown parameters matrix . In addition, the wavelength information of all FBG nodes (and not just the ones at a certain cross section) on the sensor are utilized for tip position prediction. Further, the regression model does not suffer from the accumulated errors that occur during integration of position over the length of the CDM in conventional methods, since the tip position is predicted directly. Moreover, the regression model can be trained separately on the data from fewer than three FBG fibers, to still be able to predict the CDM tip position in case of fiber failure.

Fig. 3: Experimental setup including an actuation unit and an FBG interrogator. Coordinate frames are shown: optical tracker (o), reference body (r), CDM straight pose body (s), and the base of the CDM (b).

Iii Experimental Setup and True Tip Estimation

Iii-a CDM and FBG Sensor Design

The CDM used for the experiments was previously developed for orthopedic applications such as the treatment of osteolysis [13], [14] or osteonecrosis [15] (Fig. 1). It is constructed from a NiTi tube with several notches to enable a flexible bend. The outside diameter (OD) of the CDM is mm, with an inner mm open lumen for passing tools such as those used for debridement [13]. The CDM’s thin wall contains channels for passing the actuation cables and the shape sensing units. The FBG sensor used for this study has three FBG fibers attached to a flexible NiTi wire with an OD of mm in a triangular configuration (Fig. 2). Three grooves (radially apart from each other) are engraved by laser (Potomac, USA) along the length of the wire to hold three fibers each with three FBG nodes (Technica S.A, China). Due to its relatively small OD, the NiTi wire can withstand curvatures of as small a radius as mm during bending, which is sufficient to achieve large deflections of the CDM [9].

The CDM cables are actuated with two DC motors (RE16, Maxon Motor Inc. Switzerland) with spindle drives (GP 16 A, Maxon Motor, Inc. Switzerland) on the actuation unit shown in Fig. 3. A commercial controller (EPOS 2, Maxon Motor Inc. Switzerland) is used to control the individual Maxon motors. Using libraries provided by Maxon, a custom C++ interface performs independent velocity or position control of each motor of the CDM’s actuation unit.

Iii-B CDM Tip Sensing by Optical Trackers

Optical trackers can provide 3-D position information of the reflective markers with RMS error ranging between and mm [16] with relatively high frequencies up to Hz, however, they are not suitable for tracking the tip position of the CDM intra-operatively since they require a direct line of sight. Consequently, we use an optical tracker (Polaris, NDI Inc.) for true CDM tip position during the pre-operative (off-line) regression model training, even though other sensors such as stereo cameras, etc can also be used for this purpose.

We follow a two step procedure to find the tip position of the CDM relative to its base using the optical tracker (frame o). First, we establish a coordinate frame (s) at the CDM tip when it is straight with respect to a reference body that is attached to the fixed actuation unit (frame r) using a 3-D printed rigid body placed into the open lumen of the CDM (Fig. 3):


where denotes frame j with respect to frame i during the one-time coordinate frame establishment (time ). The advantage of knowing the transformation between frames s and r is that later all the information will be reported with respect to the reference body, independent of where the optical tracker is located.

For tip tracking during CDM bending, a single spherical marker attached to a post is placed into the instrument channel of the CDM. The tracked position with respect to the optical tracker coordinate () frame can be represented with respect to the base of the CDM (frame b) by:


where is the reference body coordinate frame with respect to the optical tracker coordinate frame at its current location (in case the location of optical tracker changes). , brings the CDM tip position explained in frame s to the CDM base frame ():


where and is the length of the CDM. From Eq. 11, the CDM tip positions is known with respect to the coordinate frame at the base of the CDM (frame b).

Iv Evaluation Experiments

The CDM used in this study is designed with large deflection capabilities for enhanced dexterity for orthopedic applications. Therefore, it is essential that the tip position of the CDM is estimated accurately during large deflections, since this estimation is used as feedback for position control of the CDM’s tip during surgery [17].

To evaluate the proposed data-driven method (section II-B), the CDM was bent from straight pose to large tip deflections of mm (corresponding to of CDM’s length and curvature) and the sensor’s raw FBG wavelength data as well as the true tip position from the optical tracker were collected. The collected data set was then randomly split to training and testing data and this random split was repeated times to ensure the model’s performance is not dependant on the choice of the training data. The regression model was trained on the training data and evaluated on the unseen test data. In addition, the large deflection bending experiment was repeated times to evaluate the repeatability of the CDM tip prediction via the regression model. The same data set was used to predict the CDM’s tip position via the conventional model-dependent approach (section II-A) and the results were compared.

During the experiments, the CDM’s cables were actuated with a velocity of mm/s to obtain data with good resolution. FBG data was streamed by an optical sensing interrogator (Micron Optics sm 130) at Hz frequency, while optical tracker data (Polaris Vicra, Northern Digital Inc., Canada) was collected at Hz frequency. To this end, a thread-safe mechanism from open source c++ CISST-SAW libraries (https://github.com/jhu-cisst/cisst-saw) was used to record the data for each of these sensor sources in parallel, using a separate CPU thread for each source. All the data was time-stamped and pre-processed during the training phase to time-align the FBG and optical tracker data samples. All experiments were run on a computer with an Intel 2.3 GHz core i7 processor with 8GB RAM, running Ubuntu .

Fig. 4: Shape reconstruction and TPE using the conventional model-dependent approach. Stars represent the ground truth data from the optical tracker.
Fig. 5: TPE in each Cartesian direction using the proposed data-driven approach on the test data-set that is unseen by the regression model during training. Blue and orange represent the ground truth and estimated CDM’s tip position, respectively.

V Results and Discussion

Fig. 4 demonstrates the performance of the conventional model-dependent approaches in TPE of the CDM. In this figure, dashed lines show the CDM shape reconstructed, whereas stars represent the true tip CDM position from optical tracker data. In these experiments, the CDM was bent from a straight pose all the way to large curvatures of approximately . It is observed that the model-dependent approach can estimate the tip position relatively accurately for tip deflections below mm (corresponding to of the CDM’s length) with mean absolute error (and standard deviation) of () mm. However, for larger deflections the deviation between the estimated and true tip position increases, exhibiting larger errors in TPE of the CDM. This increase of error with larger deflections may be due to imprecise assumptions on the sensor’s and CDM’s geometries, as well as error propagation due to reconstructing shape from base to tip in the conventional model-dependent method. In addition, due to limited number of active sensing locations along the sensor, even small errors in curvature estimation for active locations closer to the base of the CDM can result in large tip estimation errors. Using conventional model-dependent approach, tip estimation error mean (and standard deviation) of () mm and () mm are observed for CDM tip deflections above mm and for all deflections, respectively.

As mentioned in section IV, the regression model is trained on a random split of the data-set ( training data and testing data) with a training time of nearly seconds. Fig. 5 demonstrates the performance of the trained model on the unseen test data-set using only the raw FBG data. The predicted tip position values in both directions of motion are shown in orange dashed lines while the ground truth tip position information from the optical tracker is displayed in blue solid line. It is observed that the tip position prediction from the regression model is following the true tip position in both directions and also for large deflections very accurately, with a mean (and standard deviation) of () mm for all deflections. Of note, to ensure the trained model’s performance is not dependant on the specific choice of training data, the training and testing phases have been performed times with completely random split of the data-set, and similar performance is observed, with a mm deviation in tip position mean absolute error.

Manipulator Tip Position
Error [mm]
Mean Std. Max
Conventional Approach
(Three Fibers Required)
1.52 0.67 3.63
Data-driven Regression
(One Fiber Used)
1.98 1.13 4.62
Data-driven Regression
(Two Fiber Used)
0.3 0.16 1.01
Data-driven Regression
(Three Fiber Used)
0.11 0.10 0.62
TABLE I: cdm tpe error comparison between the conventional sensor model-dependant method and the data-driven model using different number of fibers.

One of the advantages of the proposed data-driven regression method is that the CDM tip position can still be estimated in 3-D with less than three FBG fibers at each cross section. This can be beneficial in case one or two fibers malfunction or get damaged during the surgery. Table I summarizes the results obtained from TPE using the conventional model-dependent approach and the data-driven regression method using different number of FBG fibers. It can be concluded that the data-driven method not only improves performance compared to the conventional model-dependent method, but is still capable of estimating the tip position in absence of one or two FBG fibers as well.

Vi Conclusion

In this study we proposed a data-driven regression model for TPE of a CDM with large deflections that did not require any prior assumptions regarding the sensor geometry (FBG node location and substrate geometry), and could still predict CDM tip position with fewer than three fibers. The regression model was trained in a short amount of time pre-operatively (off-line) on a randomly chosen data-set of FBG raw wavelength data as input and true tip position from optical tracker as output. It could then predict the CDM tip position intra-operatively (on-line) using only the FBG raw wavelength values from the sensor’s current readings.

In this paper, performance of the proposed data-driven method was studied on a CDM developed primarily for orthopedic applications with large deflections. Results exhibited accuracy improvement especially in large deflections compared to the conventional model-dependent approach. This improvement could be explained as a result of making no assumptions regarding the sensor’s geometry, as well as simultaneously using all FBG nodes on the sensor to estimate the CDM tip position, rather than only a few at specific cross sections. In addition, using the data-driven approach, even with fewer than three fibers the FBG sensor could still be functional in estimating the CDM tip, in case of fiber malfunction or damage during surgery.

It should be noted the proposed method is independent of the sensor modality (e.g. optical trackers or stereo cameras) used for off-line true CDM tip position measuring that is fed to the regression model during the training phase. Future work includes study of more complex CDMs and shapes, for which nonlinear regression models or different learning techniques may be required.


  • [1] I.D. Walker, H. Choset and G.S. Chirikjian, ”Snake-Like and Continuum Robots,” in Springer Handbook of Robotics, Springer, 2016, pp. 481-498.
  • [2] R.J. Roesthuis, M. Kemp, van den Dobbelsteen, John J and S. Misra, ”Three-dimensional needle shape reconstruction using an array of fiber bragg grating sensors,” IEEE/ASME Transactions on Mechatronics, vol. 19, pp. 1115-1126, 2014.
  • [3] C. Shi, X. Luo, P. Qi, T. Li, S. Song, Z. Najdovski, T. Fukuda and H. Ren, ”Shape sensing techniques for continuum robots in minimally invasive surgery: A survey,” IEEE Transactions on Biomedical Engineering, vol. 64, pp. 1665-1678, 2017.
  • [4] H. Liu, A. Farvardin, R. Grupp, R.J. Murphy, R.H. Taylor, I. Iordachita and M. Armand, ”Shape tracking of a dexterous continuum manipulator utilizing two large deflection shape sensors,” IEEE Sensors Journal, vol. 15, pp. 5494-5503, 2015.
  • [5] R.J. Roesthuis, S. Janssen and S. Misra, ”On using an array of fiber Bragg grating sensors for closed-loop control of flexible minimally invasive surgical instruments,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pp. 2545-2551, 2013.
  • [6] R. Xu, A. Yurkewich and R.V. Patel, ”Curvature, torsion, and force sensing in continuum robots using helically wrapped FBG sensors,” IEEE Robotics and Automation Letters, vol. 1, pp. 1052-1059, 2016.
  • [7] J. Wei, S. Wang, J. Li and S. Zuo, ”Novel integrated helical design of single optic fiber for shape sensing of flexible robot,” IEEE Sensors Journal, vol. 17, pp. 6627-6636, 2017.
  • [8] A. Ebrahimi, C. He, M. Roizenblatt, N. Patel, S. Sefati, P. Gehlbach and I. Iordachita, ”Real-Time Sclera Force Feedback for Enabling Safe Robot-Assisted Vitreoretinal Surgery,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3650-3655, 2018.
  • [9] S. Sefati, M. Pozin, F. Alambeigi, I. Iordachita, R.H. Taylor and M. Armand, ”A highly sensitive fiber Bragg Grating shape sensor for continuum manipulators with large deflections,” in SENSORS, 2017 IEEE, pp. 1-3, 2017.
  • [10] S. Sefati, F. Alambeigi, I. Iordachita, M. Armand and R.J. Murphy, ”FBG-based large deflection shape sensing of a continuum manipulator: Manufacturing optimization,” in SENSORS, 2016 IEEE, pp. 1-3, 2016.
  • [11] S.C. Ryu and P.E. Dupont, ”FBG-based shape sensing tubes for continuum robots,” in Robotics and Automation (ICRA), 2014 IEEE International Conference on, pp. 3531-3537, 2014.
  • [12] S. Sefati, F. Alambeigi, I. Iordachita, R.H. Taylor and M. Armand, ”On the effect of vibration on shape sensing of continuum manipulators using fiber Bragg gratings,” in Medical Robotics (ISMR), 2018 International Symposium on, pp. 1-6, 2018.
  • [13] F. Alambeigi, S. Sefati, R.J. Murphy, I. Iordachita and M. Armand, ”Design and characterization of a debriding tool in robot-assisted treatment of osteolysis,” in Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 5664-5669, 2016.
  • [14] F. Alambeigi, S. Sefati and M. Armand, ”A Convex Optimization Framework for Constrained Concurrent Motion Control of a Hybrid Redundant Surgical System,” ArXiv Preprint arXiv:1801.06864, 2018.
  • [15] F. Alambeigi, Y. Wang, S. Sefati, C. Gao, R.J. Murphy, I. Iordachita, R.H. Taylor, H. Khanuja and M. Armand, ”A curved-drilling approach in core decompression of the femoral head osteonecrosis using a continuum manipulator,” IEEE Robotics and Automation Letters, vol. 2, pp. 1480-1487, 2017.
  • [16] Polaris Optical Tracking Systems, Northern Digital Inc., Waterloo, Canada, 2013.
  • [17] S. Sefati, R. Murphy, F. Alambeigi, M. Pozin, I. Iordachita, R. Taylor and M. Armand, ”FBG-Based Control of a Continuum Manipulator Interacting With Obstacles,” Accpeted, Intelligent Robots and Systems (IROS), 2018 IEEE/RSJ International Conference on, 2018.