Colonoscope tracking method based on shape estimation network

04/20/2020 ∙ by Masahiro Oda, et al. ∙ Aichi Institute of Technology 9

This paper presents a colonoscope tracking method utilizing a colon shape estimation method. CT colonography is used as a less-invasive colon diagnosis method. If colonic polyps or early-stage cancers are found, they are removed in a colonoscopic examination. In the colonoscopic examination, understanding where the colonoscope running in the colon is difficult. A colonoscope navigation system is necessary to reduce overlooking of polyps. We propose a colonoscope tracking method for navigation systems. Previous colonoscope tracking methods caused large tracking errors because they do not consider deformations of the colon during colonoscope insertions. We utilize the shape estimation network (SEN), which estimates deformed colon shape during colonoscope insertions. The SEN is a neural network containing long short-term memory (LSTM) layer. To perform colon shape estimation suitable to the real clinical situation, we trained the SEN using data obtained during colonoscope operations of physicians. The proposed tracking method performs mapping of the colonoscope tip position to a position in the colon using estimation results of the SEN. We evaluated the proposed method in a phantom study. We confirmed that tracking errors of the proposed method was enough small to perform navigation in the ascending, transverse, and descending colons.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

We propose a colonoscope tracking method using a shape estimation network. CT image-based diagnosis of the colon is performed as one of less invasive diagnosis methods. It is called as CT colonography. If colonic polyps or early-stage cancers are found in diagnosis, a colonoscopic examination or polypectomy is performed. In a colonoscopic examination, a physician explores in the colon to find polyps. However, understanding where the colonoscope running in the colon is difficult. A colonoscope navigation system that navigates a physician to polyp positions during a colonoscope insertion is necessary. To achieve a colonoscope navigation system, a colonoscope tracking method that finds the colonoscope position in the colon needs to be developed.

Colonoscope tracking methods were proposed by some research groups. Image-based method[4] is difficult to keep tracking when unclear colonoscopic views appear. Electromagnetic (EM) sensor will be necessary to perform tracking. A system that shows colonoscope shape in the colon using EM sensors have been developed[2, 3]. The system cannot perform navigation because it cannot map the colonoscope shape to the colon shape of a patient. A colonoscope tracking method that utilizes a CT volume and EM sensors was proposed[6]. This method obtains two curved lines representing colonoscope and colon shapes to estimate a colonoscope position on a CT volume coordinate system. However, large tracking errors were caused due to colon deformations occurred during the colonoscope insertions. To reduce this problem, estimation methods of the colon shape were proposed[5, 7]. The shape estimation network (SEN)[7]

accurately estimates colon shape utilizing temporal information. The SEN performs estimation using a neural network containing long short-term memory (LSTM) layer. Because the SEN employs a machine learning framework, it needs to be trained using data (colon and colonoscope shapes) obtained during colonoscope operations. However, the SEN was trained using data measured during operations of a colonoscope by an engineering researcher. The colonoscope operations were different from that of performed by physicians.

In this paper, we propose a colonoscope tracking method. We utilize EM sensors attached to the colonoscope and a CT volume of a patient. Colonoscope and colon shapes are obtained from them. The SEN estimates a colon shape that are deformed by a colonoscope inserted to it. To perform colon shape estimation based on real physician’s data, we trained the SEN using data obtained during physician’s colonoscope operations. We perform colonoscope position estimation utilizing the colonoscope shape and the estimated colon shape obtained from the trained SEN.

2 Colonoscope Tracking Method

2.1 Overview

We obtain two shapes representing a colonoscope and a colon. The colonoscope and colon shapes are registered to find a position in the CT volume coordinate system that corresponds to the colonoscope tip position. The colon shape changes during colonoscope insertions. We use the SEN[7] to estimate deformed colon shapes. Correspondences of points on the colonoscope shape and the estimated colon shape are established based on the deformation estimation results. A colonoscope tip position in the CT volume coordinate system is calculated from the correspondence.

2.2 Colonoscope and colon shapes

We represent the colonoscope and colon shapes as sets of points. The colonoscope shape is a set of points and directions aligned along a colonoscope. is the total number of the points in the colonoscope shape. The colon shape is a set of points aligned along a colon. is the total number of the points in the colon shape. Examples of them are found in Fig. 1.

Figure 1: SEN[7] estimates colon shape at time from colonoscope shapes in time . contains position and direction data of colonoscope shape. and are positional and directional correlations of colonoscope shape. In neural network layers, “s5x5” represents kernel size and “n72” represents number of kernel or unit.

2.3 Colon shape estimation using SEN

We use the SEN[7] to estimate colon shapes which are deformed during colonoscope insertions. The SEN is a neural network containing convolution and LSTM layers. Its input is a set of the time-series colonoscope shapes. It outputs an estimated colon shape (Fig. 1).

In the previous work[7], the SEN was trained using colonoscope and colon shapes measured during colonoscope operations performed by an engineering researcher. We measured colonoscope and colon shapes during colonoscope operations by physicians. The physician’s data are used to train the SEN. Measurement of the shapes are explained in the Sec. 3.2.

2.4 Colonoscope tip position calculation

A point in the colonoscope shape represents the colonoscope tip position. We find a point in the estimated colon shape which is closest to . We find a point in the colon shape that corresponds to . Each point in the colon and estimated colon shapes have one-to-one correspondence. As a result of these two-steps mappings, we obtain as a point in the CT volume coordinate system where the colonoscope tip located.

3 Experiments

3.1 Materials

We evaluated the proposed colonoscope tracking method in phantom-based experiments. We use a colon phantom (colonoscopy training model type I-B, Koken, Tokyo, Japan), a CT volume of the phantom, a colonoscope (CF-Q260AI, Olympus, Tokyo, Japan), an EM sensor (Aurora 5/6 DOF Shape Tool Type 1, NDI, Ontario, Canada), and a distance image sensor (Kinect v2, Microsoft, WA, USA). Experimental materials are shown in Fig. 2 (a).

(a) (b)
Figure 2: (a) experimental materials. (b) positions of 12 markers on the colon phantom.

3.2 Colonoscope and colon shapes measurement

The colonoscope shape is measured using the EM sensor. The EM sensor is strap-shaped with six sensors at its tip and points along its body. It is inserted into the colonoscope working channel. The colonoscope shape is a set of six points.

The colon shape is measured using the distance image sensor. We obtain 12 three-dimensional points on the colon phantom using the sensor. The points are aligned along the colon centerline. The colon shape is a set of the 12 points.

Both of the colonoscope and colon shapes are registered to the CT volume coordinate system. The colonoscope shape is registered using the iterative closest point (ICP) algorithm[1] by minimizing distance between the colonoscope shape and a point set on the colon centerline in the CT volume. The colon shape is registered manually.

To obtain training data of the SEN[7], we measured the colonoscope and colon shapes during colonoscope insertions to the colon phantom. Operators moved the colonoscope tip from the cecum to anus. 10 times of colonoscope movements from the cecum to anus were measured in our experiments. Two physicians (four times colonoscope movements) in the Nagoya University Hospital and an engineering researcher (six time colonoscope movements) were operated. The SEN was trained using the measured data. Figure 3 shows the configuration of experimental setup for measurement of physician’s colonoscope operation.

Figure 3: Experimental setup for measuring colonoscope operations of physicians. EM sensor and distance image sensor measure colonoscope and colon shapes during colonoscope insertions by physicians.

3.3 Definition of tracking error

We placed 12 markers on the surface of the colon phantom (Fig. 2 (b)). The markers are identifiable points both on the colon phantom and in its CT volume. We employed tracking error as a criterion of tracking performance evaluation. The tracking error is the distance between an estimated colonoscope tip position and a marker position in the CT volume, when the colonoscope tip comes closest to the marker.

3.4 Tracking and evaluation

In colonoscopic examinations, physicians observe the colon while retracting the colonoscope after its insertion up to the cecum. We assume the colonoscope tip is inserted up to the cecum when the colonoscope tracking starts. The colonoscope tracking is performed while retracting the colonoscope from the cecum to the anus.

We performed colonoscope tracking five times. An engineering researcher operated the colonoscope.

4 Results

Average, maximum, and minimum tracking errors of five times colonoscope tracking are shown in Fig. 4. Tracking errors were small in the ascending, transverse, and descending colons. Large tracking errors were observed in the sigmoid colon and rectum.

Figure 4: Tracking error at each marker. Bar graph shows average tracking error with maximum and minimum values.

5 Discussion

A physician who specializes in gastroenterology commented that tracking errors smaller than 50 mm are acceptable for colonoscope navigation to polyp positions. This is because polyps are observable from the colonoscope camera when positions of the colonoscope tip and polyps are closer than 50 mm. The proposed method achieved average tracking errors smaller than 50 mm in the ascending, transverse, and descending colons. The previous method[6] achieved it in the ascending and descending colons (under setting of mm). In the transverse colon, smaller tracking errors were observed in the proposed method. The previous method[6] does not consider colon deformation. It makes tracking errors in the transverse colon large because the transverse colon is easy to deform. Consideration of colon deformation strongly contributed to reduce tracking errors.

The tracking errors in the sigmoid colon and rectum were large. This is because the sigmoid colon and rectum have heavy deformation during the colonoscope insertions. The deformation contains considerable variation. The data used to train the SEN was not enough to provide information about deformation in these colon regions. We need to correct data that containing various deformations of the sigmoid and rectum to improve both of the colon shape estimation and tracking accuracies.

6 Conclusions

We proposed a colonoscope tracking method using the SEN. Previous tracking method caused large tracking errors due to colon deformations. In our method, the SEN estimates deformed colon shape. The SEN was trained using data obtained during colonoscope operations by physicians. The estimated colon shape is used to make correspondence between the colonoscope tip position and its position in the colon. In our phantom study, tracking errors in the transverse colon were reduced by considering colon deformations. Future work includes evaluation using more markers, reduction of tracking error in the sigmoid and rectum, and application to tracking in human colon.

Acknowledgements.
Parts of this research were supported by the MEXT/JSPS KAKENHI Grant Numbers 26108006, 17H00867, the JSPS Bilateral International Collaboration Grants, and the JST ACT-I (JPMJPR16U9).

References

  • [1] P. J. Besl and N. D. McKay (1992) A method for registration of 3-D shapes. Vol. 14, pp. 239–256. Cited by: §3.2.
  • [2] L. Y. Ching, K. Moller, and J. Suthakorn (2010) Non-radiological colonoscope tracking image guided colonoscopy using commercially available electromagnetic tracking system. Cited by: §1.
  • [3] M. Fukuzawa, J. Uematsu, S. Kono, S. Suzuki, T. Sato, N. Yagi, Y. Tsuji, K. Yagi, C. Kusano, T. Gotoda, T. Kawai, and F. Moriyasu (2015) Clinical impact of endoscopy position detecting unit (UPD-3) for a non-sedated colonoscopy. Vol. 21, pp. 4903–4910. Cited by: §1.
  • [4] J. Liu, K. R. Subramanian, and T. S. Yoo (2013) An optical flow approach to tracking colonoscopy video. Vol. 37, pp. 207–223. Cited by: §1.
  • [5] M. Oda, T. Kitasaka, K. Furukawa, R. Miyahara, Y. Hirooka, H. Goto, N. Navab, and K. Mori (2018) Machine learning-based colon deformation estimation method for colonoscope tracking. Vol. 10576, pp. 1057619–1–1057619–6. Cited by: §1.
  • [6] M. Oda, H. Kondo, T. Kitasaka, K. Furukawa, R. Miyahara, Y. Hirooka, H. Goto, N. Navab, and K. Mori (2017) Robust colonoscope tracking method for colon deformations utilizing coarse-to-fine correspondence findings. Vol. 12, pp. 39–50. Cited by: §1, §5.
  • [7] M. Oda, H. R. Roth, T. Kitasaka, K. Furukawa, R. Miyahara, Y. Hirooka, H. Goto, N. Navab, and K. Mori (2018)

    Colon shape estimation method for colonoscope tracking using recurrent neural networks

    .
    Note: (accepted) Cited by: §1, Figure 1, §2.1, §2.3, §2.3, §3.2.