1 Introduction
Fundus imaging can help to diagnose and monitor a number of diseases, such as diabetic retinopathy, glaucoma, agerelated macular degeneration [1]. Visual analysis by a trained ophtamologist can be extremely timeconsuming and hinder broad clinical application. To support this workflow, automatic segmentation [2] of the retinal vessel tree has been studies for decades, among which a vessel enhancement filter by Frangi et al. [3] is the most popular and forms the basis to various other strategies [4]
. However, this task is particularly challenging due to the complex structure of retinal vessels (e.g. branching, crossing) and low image quality (e.g. noise, artifacts, low resolution). In recent years, deep learning techniques have been exploited in the field of retinal vessel segmentation
[5, 6]. These approaches are datadriven, and tend to share a similar structure, where a classifier follows a feature extractor. For instance, Maji et al.
[5]use an autoencoder as a feature extractor, and random forests as a classifier; Tetteh et al.
[6]apply a convolutional neural network (CNN) inception model for feature extraction and another CNN model for classification. These novel, datadriven approaches perform comparably to conventional methods, but do not yield significant improvements.
In this paper, we also investigate deep learning techniques on retinal vessel segmentation. To avoid an explicit separation into “feature extractor" and “classification", we propose an alternative approach, which formulates the Frangi filter as a preweighted network (“FrangiNet"). The intuitive reasoning behind such an approach is that, by representing the Frangi filter as a preweighted neural network, subsequent training of the latter should improve segmentation quality. We aim to utilize prior knowledge of tube segmentation as a basis, and further improve it using a datadriven approach.
2 Materials and Methods
2.1 Frangi Vessel Enhancement Filter
The Frangi vesselness filter [3] is based on the eigenvalue analysis of the Hessian matrix in multiple Gaussian scales. Coarse scale structures are typically obtained by smoothing the image with a Gaussian filter where
is the standard deviation. The Hessian matrix is a square matrix of secondorder partial derivatives of the smoothed image. Therefore, it can be alternatively calculated by convolving the original image patch directly with a 2D kernel
, which is the secondorder partial derivatives of the Gaussian filter ,(1) 
The Hessian matrix is calculated as,
(2) 
The two eigenvalues of the Hessian matrix are denoted by
and , which are calculated as(3) 
A high vesselness response is obtained if and satisfy the following conditions: , and . A mathematical description of the vesselness response is presented in [3],
(4) 
where is the second order structureness, is the blobness measure, and stands for the vesselness value. are imagedependent parameters for blobness and structureness terms, and are set to 0.5, 1, respectively.
When using a Gaussian kernel with a standard deviation , vessels whose diameters equal to have the highest vesselness response. The diameter of retinal vessels in this work range from 6 to 35 pixels. Hence we choose a series of as pixels accordingly. Instead of convolving a patch with three different kernels (i. e., ), we create a threelevel resolution hierarchy [4], and convolve each level with only. The resolution hierarchy consists of the original patch and two downsampled versions, using factors 2 and 4, respectively.
2.2 Preweighted FrangiNet
Inspired by [7], we implement the multiscale Frangi filter as a neural network called FrangiNet on the basis of the previous section. The architecture of FrangiNet is described in Fig. 1. The proposed method is used to analyze image patches, with an output size set to pixels. Input patches for the three subnets are cropped as in Fig. 1 (a).
For a singlescale subnet, we first use a resize layer to downsample images to the corresponding size. Secondly, we use a 2D convolution layer with three filters to get the Hessian matrix. These filters are initialized as second partial derivatives of the twodimensional Gaussian function with . After that, a combination of mathematic operation layers are applied, based on Eqs. 3 and 4 to calculate eigenvalues and vesselness responses of this scale.
The raw vesselness scores from FrangiNet typically range from 0 to roughly 0.4. To create a binary vessel mask, a threshold is used, which is set to in this work. This means that most background pixels are squashed inbetween , and the rest few vessel pixels spread along
. To obtain a probability map, we subtract the data with the threshold
, and asymmetrically scale it, where negative values are multiplied with and positive values with so that raw scores are redistributed to . Afterwards, we apply a sigmoid layer. In this way, raw vesselness values of and are converted to and 1, respectively.2.3 Trainable FrangiNet
With the proposed network structure, we observe two sets of trainable parameters in the network: first, the covolution kernels responsible for the computation of the Hessian matrix; second, the parameters , that control the influence of structureness and blobness features. For each scale, this results in three trainable convolution kernels and two additional parameters that can be adapted during training. In FrangiNet, singlescale nets are structured in parallel and updated independently, which means in total we have nine kernels and six parameters to update during training.
A high resolution fundus (HRF) image database [8] is used in this work. The database includes 15 images of each healthy, diabetic retinopathy, and glaucomatous subjects. The HRF images are high resolution pixel RGB fundus photographs. Only the green channels are used where vascular structures manifest a good contrast [9]. The intensity values are normalized to [0, 1]. We train FrangiNet on each cartegory independently, randomly dividing the 15 datasets into 10 training, 2 validation and 3 testing sets. We also train the network with all 45 datasets, randomly taking 30 sets for training, 6 for validation and 9 for testing. As segmentation of retinal vessels from fundus images is an unbalanced classification problem, where the background pixels outnumber vessel pixels by approximately 10 to 1, dice coefficient loss [10] instead of the common cross entropy loss is used. The optimizer we utilize here is gradient descent with learning rate
and momentum 0.5. 1000 steps are used for training on each category independently, while 3000 steps are used for training on the whole database. Batch size is chosen as 250. We use python and tensorflow framework for the whole implementation.
3 Results
A quantitative comparison of the trained FrangiNets with the original Frangi filter is summarized in Table 1
. Each quality metric is calculated as an average of that of the testing datasets. After finetuning with four different training datasets, the F1 score, accuracy, precision and recall of the segmentation results on the testing datasets all get improved. Segmentation of heathy fundus images performs best, achieving a highest posttraining F1 score as 0.712. Comparing the diseased cases to the healthy cases, the F1 score has a more significant improvement after training, up to
.A comparison of the segmentation results before and after training using the healthy datasets is shown in Fig.2. Fig. 2(a) displays that FrangiNet can segment main vessels well before training. However, it only partially segments the middlesized vessels and fails to recognize most of the tiny ones. In contrast, after training, FrangiNet is able to detect the middlesized vessels well, and it also detects most of the tiny ones (Fig. 2(b)).
Dataset  F1 score  accuracy  precision  recall  

before  after  before  after  before  after  before  after  
Healthy  0.669  0.712  0.836  0.843  0.606  0.675  0.746  0.753 
Diabetic retinopathy  0.495  0.532  0.819  0.822  0.524  0.588  0.468  0.486 
Glaucomatous  0.495  0.584  0.838  0.841  0.477  0.562  0.608  0.618 
Whole dataset  0.612  0.672  0.847  0.855  0.603  0.675  0.623  0.670 
4 Discussion
In this work, we proposed to combine the prior knowlegde about retinal vessel that is encoded in the FrangiFilter with the datadriven capabilities of neural networks. We constructed a net that is equivalent to the multiscale Frangi filter. We identified the trainable parts of FrangiNet as convolutional kernels and parameters when computing vesselness. We redistributed the raw output vesselness values into a probability map and trained the net by optimizing dice coefficient loss. In experiments with high resolution fundus images of healthy and diseased patients, the trained FrangiNet performs better than the original formulation both quantitatively and qualitatively.
Future work will investigate on different network architectures for FrangiNet, or the combination of FrangiNet with other networks. We will also evaluate FrangiNet with other datasets.
References
 [1] Maji D, Santara A, Mitra P, Sheet D. Ensemble of deep convolutional neural networks for learning to detect retinal vessels in fundus images. arXiv. 2016;.
 [2] Kirbas C, Quek F. A review of vessel extraction techniques and algorithms. ACM Computing Surveys (CSUR). 2004;36(2):81–121.
 [3] Frangi AF, Niessen WJ, Vincken KL, Viergever MA; Springer. Multiscale vessel enhancement filtering. MICCAI. 1998; p. 130–137.
 [4] Budai A, Bock R, Maier A, Hornegger J, Michelson G. Robust vessel segmentation in fundus images. Int J Biomed Imaging. 2013;.
 [5] Maji D, Santara A, Ghosh S, Sheet D, Mitra P; IEEE. Deep neural network and random forest hybrid architecture for learning to detect retinal vessels in fundus images. EMBC. 2015; p. 3029–3032.

[6]
Tetteh G, Rempfler M, Zimmer C, Menze BH; Springer.
DeepFExt: Deep Feature Extraction for Vessel Segmentation and Centerline Prediction.
International Workshop on Machine Learning in Medical Imaging. 2017; p. 344–352.
 [7] Würfl T, Ghesu FC, Christlein V, Maier A; Springer. Deep learning computed tomography. Med Image Comput Comput Assist Interv. 2016; p. 432–440.
 [8] Budai A, Odstrcilik J. High Resolution Fundus Image Database; 2013.
 [9] Yin B, Li H, Sheng B, Hou X, Chen Y, Wu W, et al. Vessel extraction from nonfluorescein fundus images using orientationaware detector. Med Image Anal. 2015;26(1):232–242.
 [10] Taha AA, Hanbury A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. 2015;15(1):29.
Comments
There are no comments yet.