Segmentation and delineation of the articulated topology of retinal vessel network and structural attributes of vessels such as thickness, run length, tortuosity and branching patterns provide first level pathological cue for identifying various cardiovascular and opthalmogic diseases such as diabetes, hypertension, arteriosclerosis . Automatic analysis of retinal vessel topography assists in developing robust screening systems for diabetic retinopathy , localizing of foveal avascular region , thinning of arteries  and laser surgery . Vessel tortuosity embeds significant information about hypertensive retinopathy  while vessel diameter had been studied in connection with hypertension . Besides clinical areas, retinal topography has also been used for biometric applications .
Related Works: Unsupervised paradigms of automatic vessel detection primarily aims at designing matched filters by convolving a Gaussian kernel or its derivatives to emphasize vessel regions . Popular unsupervised multiscale approaches assign a ‘vesselness’ metric based on eigen analysis of the Hessian . Unsupervised model based techniques include active contour models and geometric model guided level sets 
. Supervised models rely on feature extraction on manually annoted ground truth images followed by classification, usually using Artificial Neural Network (ANN), Support Vector Machine (SVM) or their variants. Most common practise for feature extraction involves Gabor filter response at different strata of scale and degree. Other notable feature extraction methods include orientation analysis of gradient vector field , line operators , ridge profiles  and modelling tissue-photon interactions .
Challenges and Mitigation Strategies:
The aforementioned traditional methods rely on hand crafted low level visual features and are inept at automatically learning generalized features from training data. Designing such carefully formulated task specific features requires appreciable domain expertise. Recent paradigms of deep learning aim at unsupervised generalized feature extraction from the raw image data itself. Our work is primarily invigorated by the recent success of hierarchical feature extraction frameworks for retinal vessel segmentation[13, 14] using deep neural networks.
Contributions: We present an architecture which supports two parallel levels of ensembling of stacked denoised autoencoder (SDAE) networks. As shown in Fig. 1, first level of ensemble, also termed as ”E:Net”, is formed by training parallel SDAE networks (of same architecture) on booststrap samples while second level ensemble (Fig. 2) is realized by parallel training of two ”E:Net”s of different architecture. Initial architecture of any deep network is usually intuitive and needs to be fine tuned based on classification accuracy. We explore the possibility of leveraging performance of different architectured networks within the context of ensemble learning. We study various techniques for combining decisions of individual members of the ensemble and we show that using only simple SoftMax classifier we outperform the recent deep learning based method 
which uses random forest classifier after unsupervised feature extraction. For mitigating class imbalance between vessel and background, we present a set of smart sampling procedures for creating a database congenial for training SDAE.
The rest of the paper is organized as follows. The problem statement is formally defined in §II and the detailed solution is presented in §III. In Section §IV we validate and compare our results with state-of-the-art vessel segmentation algorithms. Finally, we conclude the paper in §V with a summary of aptness of our ensemble learning of deep networks for automatically generating diversified dictionary kernels for medical image analysis.
Ii Problem Statement
Given a color fundus image,
, our objective is to assign a probability to each pixel (x,y) such that its neighbourhood,, centred at belongs to either of the classes, . Specifically, given a training set, , we wish to formulate a function, , whose response gives us . In this paper we learn the function by hierarchical feature extraction of using our proposed fusion of SDAE ensemble networks.
Iii Proposed Approach
Iii-a Image Preprocessing, sampling procedure and database formation
Given a RGB fundus image, , we first extract the green channel image, , because it has been reported that the vascular structures manifest best contrast in green channel . CLAHE is used for compensating irregular illuminations.
shows an exemplary manually annotated vessel network used for extracting training patches. There exists significant class imbalance between background and vessel pixels. It is a challenging task to train classifiers in presence of highly skewed class distributions. We propose a set of intuitive sampling strategies to mitigate this problem. Let the ground truth binary image be . For sampling vessel patches, we first skeletonize the image to . Image skeletonization helps in removing the boundary foreground (vessel) pixels but retains the overall topological structure and thereby prevents redundant foreground sampling. Vessel training patches, , are uniformly sampled from at those coordinates for which . For sampling background pixels, we first morphologically dilate to by a square structural element of dimension . Naively sampling at generates samples which are very near to original vessels and thus the neighbourhood encompass considerable region of vessel tissue region. Dilation operations thickens the vessels and thus sampling with ensures that the background training patches are well separated from original vessel regions. For each training image , if represents the set of vessel patches and represents set of background patches, we uniformly sample samples from to form . As a final measure to enhance classification performance, an autoencoder is trained on a dataset with alternating patches from vessel and background class.
Iii-B Unsupervised layerwise prelearning using SDAE
For automated feature discovery we have used stacked denoised autoencoders which are analogous in architecture to traditional multilayer perceptron (MLP) networks but in DAE, interconnection weights are learnt sequentially through unsupervised layerwise training (Refer to Sec. III-B for more details). An autoencoder is basically a three layer fully connected network with one hidden layer which stores a compressed representation of input , and outputs, , which is an approximation of . Let denote number of layers, denote number of nodes in layer , denote weight between node of layer to node of layer and is sigmoidal activation. According to 
, loss function of sparse autoencoder is given by,
The first term tries to minimize the discrepancy between input and predicted vector, second term is meant for L2 regularization to prevent over fitting and the last term promotes sparsity within the network. is sparsity penalty, is cardinality of training sample space, is a sparseness parameter, is th expected activation of node in hidden layer, i.e., , where, is the activation of node in hidden layer. DAE is a specialized version of autoencoder in which we incorporate random additive noise at input side to transform to . Loss function in Eq. 1 is minimized by back propagation and the parameters and are updated using L-BFGS (Limited Memory Broyden-Fletcher-Goldfarb-Shanno). After this first step of pre-training, a stacked DAE is realized by treating the hidden layer node activations as input to a second DAE and retraining the second DAE only. After second phase of pre-training we discard the last output layer and insert a simple SoftMax classification layer with only a single node which provides the probability,
. Now, this SoftMax layer is used for fine tuning the entire stacked architecture similar to supervised MLP setting.
First level ensemble formation Our first level of ensemble formation is inspired by success of bagging  in ensemble learning. Given an original set, , of training examples, form sets, , ,.. , each of cardinality by randomly sampling samples from with replacement. We denote as a SDAE of depth trained on and collection of such SDAEs is termed as ”E:Net”. In Fig. 1 and 4 and we have used and interchangeably. As shown in Fig. 1, during classification, each produces a probabilistic output, . We used multiple strategies to fuse the probability maps from each .
where ; is the accuracy rate of on cross validation set.
Second level ensemble formation A SDAE (of depth 2) with SoftMax output is characterized by a network architecture, where is input dimension, is number of classes and denote number of nodes in first and second hidden layer respectively. We induce further diversification in learning by parallel training of two ” E:Net”s ( E:Net(1) and E:Net(2)) of different architecture (refer to Fig. 2), using different values of and . Decisions from each ”E:Net(i)” is merged by a convex weighted average; the weight being proportional to its accuracy on validation set.
Iv Results and Discussions
We evaluate the performance of our algorithm on the popular DRIVE dataset  and compare our results with other state-of-the-art vessel segmentation methods. In our experiment we have used an architecture of ‘576-400-100-2’ for ” E:Net(1)” and ‘576-200-50-2’ for ” E:Net(2)”. We have used , , ; this setting yields the maximum accuracy averaged over test examples (Refer Table II
). Pre-training and finetuning of each SDAE is done for 700 epochs.
In Fig. 4 we show some exemplary visual dictionary kernels learnt by and of E:Net. It is evident, specially for Stage 1, that training on bootstrap samples encourages the ensemble to learn diversified kernels.
We use the two standard metrics, viz., maximum average accuracy and Cohen’s Kappa agreement coefficient for comparing our results. In Table I we first compare the performance of individual ” E:Net” at first level of ensemble using different fusion strategies as delineated in Sec. 4.
|”E:Net”||Fusion||Max. Avg. Accuracy||Kappa|
|Min (Eq. 2)||0.932||0.687|
|1||Max (Eq. 3)||0.928||0.679|
|Average (Eq. 4)||0.908||0.654|
|Weighted Average (Eq. 5)||0.948||0.698|
|Method||Max. Avg. Accuracy ()||Kappa Agreement|
|Second human observer||0.947 (0.048)||0.758|
|Maji et al. ||0.932 (–)||0.628|
|Roy et al. ||0.912 (0.026)||0.618|
|Sigurosson et al. ||0.942 (0.010)||0.708|
|Yin et al. ||0.932 (–)||-|
|Sheet et al. ||0.976 (–)||0.821|
|Chaudhuri et al.||0.877 (0.0232)||0.33|
|Jiang et al.||0.921 (0.0076)||0.639|
|Martinez-Perez et al.||0.918 (0.0240)||0.638|
|Fraz et al.||0.948 (-)||-|
|Zana et al. ||0.937 (0.0077)||0.697|
We see that the weighted average method manifests best classification accuracy for both the first level ensemble networks. For fusing decision at second level of ensemble, we first generate the vessel probability maps by weighted average voting from both the level one ”E:Net”s and average them to yield the final soft classification output (see Fig. 2which maximizes the .
In Table II we compare our results with state-of-the-art competing algorithms. Second column shows the maximum average accuracy along with the standard deviation. Comparison of Table I and II proves that second level ensemble learning further enhances the classification accuracy and thus justifies our approach. We achieve maximum average accuracy of 0.953 with standard deviation of only 0.003 and Kappa agreement coefficient of 0.709. It is encouraging to observe that our proposed unsupervised feature discovery based model superceeds the human observer in terms of accuracy and standard deviation. Fig. 5 manifests the efficacy of our model in detecting both course and fine retinal vessels. Top row magnifies the optic nerve region where many vessels merge together and bundle up and segmentation becomes difficult but our algorithm performs appreciably in such region. Our proposed method also achieves high accuracy in segmenting sparsely distributed fine blood vessel (bottom row).
In this paper we presented a deep neural ensemble network architecture for retinal vessel segmentation. We have observed that miscellany of training space and architecture generated diversified dictionary of visual kernels for vessel detection. Each kernel is responsible for identifying a specific orientation of vessel. Learning diversified kernels thus enhances the representation prowess of our ensemble. Experimental validations suggest that our unsupervised layerwise feature discovery based model is not only highly accurate but also reliable and consistent. Future improvements might focus on selecting a better threshold from the proabability maps. Another promising direction is to use multiview ensemble learning by extracting features from each stage of a stacked autoencoder.
-  J. J. Kanski and B. Bowling, Clinical ophthalmology: a systematic approach. Elsevier Health Sciences, 2011.
-  T. Teng, M. Lefley, and D. Claremont, “Progress towards automated diabetic ocular screening: a review of image analysis and intelligent systems for diabetic retinopathy,” Med. & Biol. Eng. & Comput., vol. 40, no. 1, pp. 2–13, 2002.
-  A. Haddouche, M. Adel, M. Rasigni, J. Conrath, and S. Bourennane, “Detection of the foveal avascular zone on retinal angiograms using markov random fields,” Digit. Signal Process., vol. 20, no. 1, pp. 149–154, 2010.
-  E. Grisan and A. Ruggeri, “A divide et impera strategy for automatic classification of retinal vessels into arteries and veins,” in EMBC, vol. 1. IEEE, 2003, pp. 890–893.
-  M. Foracchia, E. Grisan, and A. Ruggeri, “Extraction and quantitative description of vessel features in hypertensive retinopathy fundus images,” in Book Abstr. 2nd Intern. Workshop on Comput. Assist. Fundus Image Anal., vol. 6, 2001.
-  J. Lowell, A. Hunter, D. Steel, A. Basu, R. Ryder, and R. L. Kennedy, “Measurement of retinal vessel widths from fundus images based on 2-d modeling,” IEEE Tran. Med. Imaging, vol. 23, no. 10, pp. 1196–1204, 2004.
-  C. Mariño, M. G. Penedo, M. Penas, M. J. Carreira, and F. Gonzalez, “Personal authentication using digital retinal images,” Pattern Anal. & Appl., vol. 9, no. 1, pp. 21–33, 2006.
-  C. Köse, C. İki et al., “A personal identification system using retinal vasculature in retinal fundus images,” Expert Syst. with Appl., vol. 38, no. 11, pp. 13 670–13 681, 2011.
-  R. J. Winder, P. J. Morrow, I. N. McRitchie, J. Bailie, and P. M. Hart, “Algorithms for digital image processing in diabetic retinopathy,” Computerized Med. Imaging & Graphics, vol. 33, no. 8, pp. 608–622, 2009.
-  M. Niemeijer, J. Staal, B. Ginneken, M. Loog, and M. Abramoff, “Drive: digital retinal images for vessel extraction,” 2004.
-  A. Hoover, V. Kouznetsova, and M. Goldbaum, “Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response,” IEEE Tran. Med. Imaging, vol. 19, no. 3, pp. 203–210, 2000.
D. Sheet, S. P. K. Karri, S. Conjeti, S. Ghosh, J. Chatterjee, and A. K. Ray, “Detection of retinal vessels in fundus images through transfer learning of tissue specific photon interaction statistical physics,” inISBI. IEEE, 2013, pp. 1452–1456.
-  D. Maji, A. Santara, S. Ghosh, D. Sheet, and P. Mitra, “Deep neural network and random forest hybrid architecture for learning to detect retinal vessels in fundus images,” in EMBC. IEEE, 2015, pp. 3029–3032.
-  A. Guha Roy and D. Sheet, “Dasa: Domain adaptation for stacked autoencoders,” in ACPR, 2015.
-  B. Yin, H. Li, B. Sheng, X. Hou, Y. Chen, W. Wu, P. Li, R. Shen, Y. Bao, and W. Jia, “Vessel extraction from non-fluorescein fundus images using orientation-aware detector,” Med. Image Anal., vol. 26, no. 1, pp. 232–242, 2015.
-  M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera, “A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches,” IEEE Tran. Syst. Man & Cybernetics, vol. 42, no. 4, pp. 463–484, 2012.
-  J. Staal, M. D. Abràmoff, M. Niemeijer, M. A. Viergever, and B. Van Ginneken, “Ridge-based vessel segmentation in color images of the retina,” IEEE Tran. Med. Imaging, vol. 23, no. 4, pp. 501–509, 2004.
-  P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” JMLR, vol. 11, pp. 3371–3408, 2010.
-  N. A. H. Mamitsuka, “Query learning strategies using boosting and bagging,” in ICML, vol. 1. Morgan Kaufmann Pub, 1998.
-  M. Niemeijer, J. Staal, B. van Ginneken, M. Loog, and M. D. Abramoff, “Comparative study of retinal vessel segmentation methods on a new publicly available database,” in SPIE Med. Imaging, 2004, pp. 648–656.
-  E. M. Sigurosson, S. Valero, J. A. Benediktsson, J. Chanussot, H. Talbot, and E. Stefansson, “Automatic retinal vessel extraction based on directional mathematical morphology and fuzzy classification,” Pattern Recognit. Lett., vol. 47, pp. 164–171, 2014.
-  M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “An ensemble classification-based approach applied to retinal blood vessel segmentation,” IEEE Tran. Biomed. Engg., vol. 59, no. 9, pp. 2538–2548, 2012.