Brain-inspired robust delineation operator

11/26/2018 ∙ by Nicola Strisciuglio, et al. ∙ University of Groningen 0

In this paper we present a novel filter, based on the existing COSFIRE filter, for the delineation of patterns of interest. It includes a mechanism of push-pull inhibition that improves robustness to noise in terms of spurious texture. Push-pull inhibition is a phenomenon that is observed in neurons in area V1 of the visual cortex, which suppresses the response of certain simple cells for stimuli of preferred orientation but of non-preferred contrast. This type of inhibition allows for sharper detection of the patterns of interest and improves the quality of delineation especially in images with spurious texture. We performed experiments on images from different applications, namely the detection of rose stems for automatic gardening, the delineation of cracks in pavements and road surfaces, and the segmentation of blood vessels in retinal images. Push-pull inhibition helped to improve results considerably in all applications.



There are no comments yet.


page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The delineation of elongated structures is a fundamental process in image processing and computer vision, for various applications: detection of rose stems for automatic gardening robotics, crack delineation in roads or walls, segmentation of blood vessels in medical images, segmentation of roads and rivers in aerial images and so on. In these applications, images usually contain large amounts of background noise and spurious texture, which cause segmentation errors 

[1, 2, 3].

In this paper, we present a novel filter, inspired by the push-pull inhibition that is exhibited by some neurons in area V1 of the primary visual cortex. We construct a filter that has two components, an excitatory and an inhibitory one, based on the existing model of neurons with excitatory receptive fields in area V1, called CORF [4], whose implementation is known as B-COSFIRE and shown to be effective for the delineation of blood vessels in medical images [5, 6], also in combination with machin learnig techniques [7]. We name the proposed filter RUSTICO, which stands for RobUST Inhibition-augmented Curvilinear Operator, and show how push-pull inhibition contributes to strenghten the robustness with respect to noise and spurious texture in the delineation of elongated patterns. The aim of this work is to demonstrate how inspiration from neurophysiological findings can be used to design effective algorithms, on which one can build more complex systems.

State-of-the-art approaches for the delineation of curvilinear patterns in images were recently reviewed in [8]. Fundamental methods are based on a parametric formulation of the pattern of interest, namely line-like structures. The Hough transform, for instance, projects an input image onto a parameter space (slope and bias) in which linear segments are easier to detect. The disadvantage of parametric methods is that they require a strict mathematical model of the patterns of interest, and different shapes require different formulations.

Other methods are based on filtering or mathematical morphology, such as the Frangi detector that employs multi-scale local derivatives [9]. Matched filters, which model the profile of the elongated patterns with 2D Gaussian kernels, were proposed in [10]. Combination of different techniques, such as Frangi filters and Gabor Wavelets, was also studied [11]. Mathematical morphology techniques assume a-priori knowldege about the geometry of the patterns of interest [12], such as size, orientation and width [13], or concavity [14]. Recently, a method called RORPO, based on morphological path operators was proposed for the delineation of 2D and 3D curvilinear patterns [15].

Point processes were also employed to segment networks of elongated structures, which are considered as complex sets of linear segments [2]. They were also combined with Gibbs models [16], Monte-Carlo simulation [17], and graph-based representations with topological information about the line networks [18]. These methods are computationally very expensive, not being suitable for real-time or high resolution image processing.

Machine learning techniques have also been investigated to perform pixel-wise segmentation of elongated patterns. Early approaches in [19] and [20]

constructed pixel-wise feature vectors with multi-scale Gaussian and Gabor wavelet features, respectively, or with the responses of a bank of ridge detectors 


. Invariant moments were also used to describe the pixel characteristics in 


. More recently, Convolutional Neural Networks (CNNs) gained particular popularity in computer vision, for many tasks including segmentation. In 

[23], for instance, image patches containing lines were used to train a CNN for the segmentation of blood vessels in medical images. More general architectures for segmentation were proposed, such as U-Net [24] and SegNet [25]. CNNs are supervised approaches, employ a large amount of filters at different stages and usually achieve high segmentation performance, but require large amounts of labeled training data to learn effective models and are computationally very expensive, requiring GPU hardware.

The approach that we introduce in this work is unsupervised and, hence, it is not appropriate to compare it with the performance of CNNs, but rather to demonstrate how we can make use of neuro-physiological evidence about the functioning of the visual system to improve image processing and computer vision algorithms. We demonstrate the effectiveness of the push-pull inhibition phenomenon and the improved robustness of the proposed filter in three applications were spurious textures are present, namely delineation of rose stems for garden robotics, detection of cracks in road surfaces and segmentation of blood vessels in retinal images.

The paper is organized as follows. In Section 2, we describe the proposed implementation of the push-pull inhibition mechanism, and the data sets used for the experiments in Sections 3. We present the results that we achieved, and provide a discussion of the results and comparison with those obtained by existing methods in Section 4. We draw conclusions in Section 5.

2 Method

The main idea of RUSTICO is the design of an operator selective for curvilinear patterns with push-pull inhibition. This type of inhibition is known to suppress responses to spurious texture and thus emphasizing more the detection of linear structures. In practice, RUSTICO takes input from two types of COSFIRE filters of the type introduced in [5], one that gives excitatory input and the other that acts as inhibitory component. We compute the response of RUSTICO by subtracting the (weighted) response of the inhibitory component from the excitatory one.

2.1 B-COSFIRE filter

The COSFIRE filter approach is trainable, in that the selectivity of the filter is determined from an automatic configuration procedure that analyzes a given prototype pattern. In [5], a bar-like synthetic prototype pattern was used and the resulting filter responded strongly to blood vessels in retinal images. In [4], instead, an edge prototype pattern was used for configuration of a COSFIRE filter that was very effective for contour detection.

The automatic configuration of a COSFIRE filter involves two steps: first, the determination of keypoints in the given prototype pattern that we illustrate in Fig. 1 by means of a system of concentric circles and linear filtering with difference-of-Gaussians (DoG) functions and secondly, the description of such keypoints in terms of four parameters. These parameters include the polarity

and standard deviation

, which describe the type (center-on or center-off) along with the spread of the outer Gaussian function111The standard deviation of the inner Gaussian function is ) of the DoG that gives the maximum response in the concerned keypoint. The other two parameters are the distance and polar angle of the keypoint with respect to the center of the prototype. We define a COSFIRE filter as a set of 4-tuples .

(a) (b)
Figure 1: (a) A prototype line and a (b) sketch of the configuration process.

The response of a B-COSFIRE filter, denoted by

, is computed by first calculating an intermediate feature map for each tuple followed by combining all features maps by geometric mean. The feature map of tuple

is computed in four steps: firstly, convolve the image with a DoG function whose polarity is and standard deviation , secondly rectify

the output by a rectification linear unit (ReLU) that sets to zero all negative responses, thirdly

blur the thresholded response map by a Gaussian function whose standard deviation where and

are hyperparameters which we set empirically, and lastly

shift the blurred responses by the vector .

To configure a B-COSFIRE filter which is selective for the same prototype but rotated by a given offset , one can construct a new filter by manipulating the angular parameter in all tuples of the filter : . This mechanism is required to achieve tolerance to rotation. For more details on COSFIRE filters we refer the reader to [5].

2.2 Push-pull inhibition

Push-pull inhibition is a phenomenon that has been observed in many simple cells [26]. It is thought that an interneuron inhibits the response of the simple cell to which it is connected. The receptive field of the interneuron is typically larger than that of the simple cell. The effect is that the response of the simple cell is suppressed for a stimulus with preferred orientation but with contrast opposite of the preferred one. We model the interneuron by another COSFIRE filter which we denote by and define it as where is a weighting factor that controls the size of the afferent DoG functions. The response of the inhibitory COSFIRE filter is computed with the same procedure described above.

2.3 Rustico

We define an orientation-selective RUSTICO as a pair and compute its response by combining the excitatory and inhibitory inputs with a linear function:


where is the weighting or strength of the inhibitory component, which we determine experimentally, and indicates the ReLU function.

In order to configure a RUSTICO filter that is tolerant to rotations we consider various pairs of excitatory and inhibitory COSFIRE filters that are selective for different orientations and then combine their response maps. Formally, we denote by a multi-orientation RUSTICO and define it as a set . Finally, the multi-orientation RUSTICO response is achieved by taking the maximum superposition of the response maps corresponding to all pairs in the set :


3 Materials

We tested the performance of the proposed operator with push-pull inhibition on different data sets, namely the TB-roses-1, the CrackTree206 [27] and the DRIVE [21] data sets. We show example images from the three data sets, together with the corresponding ground truth delineation maps in Fig. 2.

The TB-roses-1 data set is composed of images, which we recorded in a real garden in the context of the TrimBot2020 project [28]. It is designed for testing algorithms for delineation of rose branches in applications of gardening robotics. The images have resolution of pixels and are provided together with two ground truth images, one indicating the centerline of the rose branches and the other marking the whole segmented branches. The data set is publicly available222

The CrackTree206 data set is composed of images of road surface, taken with an RGB camera at resolution pixels. The images contain spurious texture around the road cracks, due to the intrisic composition of the asphalt. This makes the delineation of cracks a hard task, since the size and contrast of the cracks are very similar to the one of the textured background. The images are provided with manually labeled images that delineate the center-line of the cracks and serve as ground truth for performance evaluation.

The DRIVE data set of retinal fundus images is divided into a training and a test set, both containing images at resolution pixels. The images are recorded with a fundus camera with field of view and are provided with manually labeled ground truth images from two different observers. Similar to existing works, we used the ground truth of the first observer as gold standard for the evaluation of our method.

4 Experiments

4.1 Evaluation

For the rose stem and road crack center-line detection, we compute the precision (Pr), recall (Re) and F-score (F). For the computation of these metrics, we consider a certain amount of pixel distance

to account for tolerance in the position of the detected center-line with respect to the position in the ground truth images [27]. We used for the TB-roses-1 data set and for the CrackTree206 data set (as reported in [27]

). We compute the evaluation metrics by thresholding the output of RUSTICO with different values of threshold

, ranging from to in steps of and report the results for the value of the threshold that contributes to the highest average F-score on the considered data set.

Figure 2: Example images from the considered data sets, together with the corresponding ground truth images.

In the case of retinal vessel delineation, we evaluate the performance of the proposed method by computing the Matthews correlation coefficient (MCC) and the Connectivity-Area-Length (CAL) measure [29]. The MCC is a reliable measure of accuracy for two-class classification problems where the cardinality of the classes is unbalanced. A value of indicates perfect classification, while and correspond to random and completely wrong classification, respectively.

Pixel-wise comparison of the output of delineation algorithms with respect to ground truth images is subject to various problems. For instance, a displacement of the segmented image of one pixel in any direction would cause a substantial reduction of performance results. Moreover, evaluation of the performance against different ground truth images causes disparate results. The CAL was demonstrated to be robust to such issues and to be in accordance with perceptual quality of the segmentation output [29]. The CAL measure is computed as the product of three measures of connectivity, area and length of the segmented line networks compared with the corresponding ground truths. Each of the single measures has values between 0 and , with indicating complete difference, while representing a perfect match with the ground truth. For further details about the computation of the CAL metric we refer the reader to [29].

4.2 Results and discussions

In Table 1, we report the results achieved by the proposed method, in comparison with those of the original COSFIRE filter. For the three considered applications, the push-pull inhibition mechanism that we embedded in the RUSTICO filter contributed to a substantial improvement of the delineation output. We achieved an increase of the value of the performance measures that is statistical significant (TB-roses: ; CrackTree206: : DRIVE: ). The improvement of results is evident from Fig. 3, where we show the precision-recall curves achieved by the RUSTICO (solid line) and COSFIRE (dashed line) filters on the TB-roses-1 (Fig. 3a) and the CrackTree206 (Fig. 3b) data sets. The curves show a substantial improvement of performance of RUSTICO with respect to COSFIRE in applications with images containing noise and spurious textures. For the CrackTree206 data set, we report the results achieved by existing methods: COSFIRE[30] (), SegExt (), Canny (), global pb () and pbCGTG (). These results, except for those of the CrackTree [27] algorithm with and without pre-processing ( and , respectively), although specifically designed to deal with the characteristics of the concerned images, are considerably lower than those of RUSTICO ().

TB-roses-1 CrackTree206 DRIVE
# images 35 206 20
Table 1: Comparison of the performance of the COSFIRE filter (C) and the proposed RUSTICO (R) on the TB-roses, CrackTree206 and DRIVE data sets. As quantitative measurements, for the former two data sets we use the F-score (F), while for the latter data set we use the CAL metric. The filters have different parameters according to the data set (TB-roses: ; CrackTree206: : DRIVE: ). The parameters and are specific of RUSTICO and not used for COSFIRE. (no GSR)CrackTree
(a) (b)
Figure 3: Precision-Recall curves achieved by RUSTICO (solid line) and COSFIRE (dashed line) on the (a) TB-roses-1 and (b) CrackTree data sets, together with the results achieved by existing approaches.

In Table 2, we report the quantitative comparison of the results achieved by RUSTICO with respect to those obtained by existing approaches on the DRIVE data set of retinal images. Among the methods based on filtering and mathematical morphology (above the middle line), RUSTICO achieves the best MCC and CAL measures with high statistical significance. We computed the performance of existing methods for which the segmentation output is publicly available. In the case of RORPO, we run experiments by varying the parameters with a grid search and reported the best obtained values. Methods based on machine learning perform generally better on pixel-wise classification, but show lower or comparable performance with that of RUSTICO in terms of CAL.

Method MCC CAL
Jiang et al. [31]
Perez et al. [13]
RORPO [15]
Zana et al. [32] -
Staal et al. [21]
Niemeijer et al. [19] -
FC-CRF [33] -
UP-CRF [33]
Table 2: Comparison of the results achieved by RUSTICO on the DRIVE data set with those obtained by existing method. The sign - indicates no statistical difference, while and indicate that the corresponding results are statistically higher than those of RUSTICO with significance level and , respectively.

5 Conclusions

We presented a new method for the delineation of elongated patterns in images with spurious texture, named RUSTICO, that incorporates a push-pull inhibition mechanism operated by some neurons in area V1 of the visual cortex. RUSTICO takes input from two COSFIRE filters with opposite polarity and responds to elongated patterns also when they are surrounded by noise. We demonstrated how the findings of neuro-physiological studies of the visual system into image processing algorithms can be used to design more robust algorithms. The push-pull inhibition included in RUSTICO contributed to a statistically significant improvement with respect to existing methods in applications of delineation of rose stems for automatic gardening, detection of cracks in road surfaces and segmentation of blood vessels in medical images. We created and made available a data set of labeled images to test algorithm for segmentation of rose stems.


This research received funding from the EU H2020 research and innovation framework (grant no. 688007, TrimBot2020).


  • [1] Zhang, L., Zhang, Y., Wang, M., Li, Y.: Adaptive river segmentation in sar images. Journal of Electronics 26(4) (2009) 438–442
  • [2] Lacoste, C., Descombes, X., Zerubia, J.: Point processes for unsupervised line network extraction in remote sensing. IEEE Trans. Pattern Anal. Mach. Intell 27(10) (2005) 1568–1579
  • [3] Strisciuglio, N., Petkov, N.: Delineation of line patterns in images using b-cosfire filters. In: IWOBI. (2017) 1–6
  • [4] Azzopardi, G., Petkov, N.: A CORF computational model of a simple cell that relies on LGN input outperforms the Gabor function model. Biological Cybernetics 106(3) (2012) 177–189
  • [5] Azzopardi, G., Strisciuglio, N., Vento, M., Petkov, N.: Trainable cosfire filters for vessel delineation with application to retinal images. Med Image Anal 19(1) (2015) 46 – 57
  • [6] Strisciuglio, N., Azzopardi, G., Vento, M., Petkov, N.: Unsupervised delineation of the vessel tree in retinal fundus images. In: VIPIMAGE. (2015) 149–155
  • [7] Strisciuglio, N., Azzopardi, G., Vento, M., Petkov, N.: Supervised vessel delineation in retinal fundus images with the automatic selection of B-COSFIRE filters. Mach. Vis. Appl. (2016) 1–13
  • [8] Bibiloni, P., González-Hidalgo, M., Massanet, S.: A survey on curvilinear object segmentation in multiple applications. Pattern Recognition 60 (2016) 949 – 970
  • [9] Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In Wells, W.M., Colchester, A., Delp, S., eds.: MICCAI 1998. (1998) 130–137
  • [10] Al-Rawi, M., Qutaishat, M., Arrar, M.: An improved matched filter for blood vessel detection of digital retinal images. Computer in biology and medicine 37(2) (2007) 262–267
  • [11] Oliveira, W.S., Teixeira, J.V., Ren, T.I., Cavalcanti, G.D.C., Sijbers, J.: Unsupervised retinal vessel segmentation using combined filters. PLoS ONE 11(2) (02 2016) 1–21
  • [12] Mendonca, A.M., Campilho, A.: Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction. IEEE Trans. Med. Imag. 25(9) (2006) 1200–1213
  • [13] Martinez-Pérez, M.E., Hughes, A.D., Thom, S.A., Bharath, A.A., Parker, K.H.: Segmentation of blood vessels from red-free and fluorescein retinal images. Med Image Anal 11(1) (2007) 47–61
  • [14] Lam, B., Gao, Y., Liew, A.C.: General retinal vessel segmentation using regularization-based multiconcavity modeling. IEEE Trans. Med. Imag. 29(7) (2010) 1369–1381
  • [15] Merveille, O., Talbot, H., Najman, L., Passat, N.: Curvilinear structure analysis by ranking the orientation responses of path operators. IEEE Trans. Pattern Anal. Mach. Intell 40(2) (2018) 304–317
  • [16] Lafarge, F., farb, G.G., Descombes, X.:

    Geometric feature extraction by a multimarked point process.

    IEEE Trans. Pattern Anal. Mach. Intell 32(9) (2010) 1597–1609
  • [17] Verdié, Y., Lafarge, F. In: Efficient Monte Carlo Sampler for Detecting Parametric Objects in Large Scenes. (2012) 539–552
  • [18] Türetken, E., Benmansour, F., Andres, B., Głowacki, P., Pfister, H., Fua, P.:

    Reconstructing curvilinear networks using path classifiers and integer programming.

    IEEE Trans. Pattern Anal. Mach. Intell. 38(12) (2016) 2515–2530
  • [19] Niemeijer, M., Staal, J., van Ginneken, B., Loog, M., Abramoff, M.: Comparative study of retinal vessel segmentation methods on a new publicly available database. In: Proc. of the SPIE - The International Society for Optical Engineering. (2004) 648–56
  • [20] Soares, J.V.B., Leandro, J.J.G., Cesar, Jr., R.M., Jelinek, H.F., Cree, M.J.: Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans. Med. Imag. 25(9) (2006) 1214–1222
  • [21] Staal, J., Abramoff, M., Niemeijer, M., Viergever, M., van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imag. 23(4) (2004) 501–509
  • [22] Marin, D., Aquino, A., Emilio Gegundez-Arias, M., Manuel Bravo, J.: A New Supervised Method for Blood Vessel Segmentation in Retinal Images by Using Gray-Level and Moment Invariants-Based Features. IEEE Trans. Med. Imag. 30(1) (2011) 146–158
  • [23] Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imag. 35(11) (2016) 2369–2380
  • [24] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: MICCAI. (2015) 234–241
  • [25] Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2017)
  • [26] Taylor, M.M., Sedigh-Sarvestani, M., Vigeland, L., Palmer, L.A., Contreras, D.: Inhibition in simple cell receptive fields is broad and off-subregion biased. Journal of Neuroscience 38(3) (2018) 595–612
  • [27] Zou, Q., Cao, Y., Li, Q., Mao, Q., Wang, S.: Cracktree: Automatic crack detection from pavement images. Pattern Recognit Lett 33(3) (2012) 227 – 238
  • [28] Strisciuglio, N., Tylecek, R., Blaich, M., Petkov, N., Bieber, P., Hemming, J., van Henten, E., Sattler, T., Pollefeys, M., Gevers, T., Brox, T., Fisher, R.B.: Trimbot2020: an outdoor robot for automatic gardening. In: 50th International Symposium on Robotics. (2018)
  • [29] Gegundez-Arias, M.E., Aquino, A., Bravo, J.M., Marin, D.: A function for quality evaluation of retinal vessel segmentations. IEEE Trans. Med. Imag. 31(2) (Feb 2012) 231–239
  • [30] Strisciuglio, N., Azzopardi, G., Petkov, N.: Detection of curved lines with b-cosfire filters: A case study on crack delineation. In: Computer Analysis of Images and Patterns. (2017) 108–120
  • [31] Jiang, X., Mojon, D.: Adaptive local thresholding by verification-based multithreshold probing with application to vessel detection in retinal images. IEEE Trans. Pattern Anal. Mach. Intell. 25(1) (2003) 131–137
  • [32] Zana, F., Klein, J.: Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation. IEEE Trans. Med. Imag. 10(7) (2001) 1010–1019
  • [33] Orlando, J.I., Prokofyeva, E., Blaschko, M.B.: A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Trans Biomed Eng 64(1) (2017) 16–27