A Convolution-Free LBP-HOG Descriptor For Mammogram Classification

by   Zainab Alhakeem, et al.

In image based feature descriptor design, an iterative scanning process utilizing the convolution operation is often adopted to extract local information of the image pixels. In this paper, we propose a convolution-free Local Binary Pattern (CF-LBP) and a convolution-free Histogram of Oriented Gradients (CF-HOG) descriptors in matrix form for mammogram classification. An integrated form of CF-LBP and CF-HOG, CF-LBP-HOG, is subsequently constructed in a single matrix formulation. The proposed descriptors are evaluated using a publicly available mammogram database. The results show promising performance in terms of classification accuracy and computational efficiency.


page 1

page 2

page 3

page 4

page 5


Face Retrieval using Frequency Decoded Local Descriptor

The local descriptors have been the backbone of most of the computer vis...

Towards Automated Biometric Identification of Sea Turtles (Chelonia mydas)

Passive biometric identification enables wildlife monitoring with minima...

FWLBP: A Scale Invariant Descriptor for Texture Classification

In this paper we propose a novel texture recognition feature called Frac...

A new Local Radon Descriptor for Content-Based Image Search

Content-based image retrieval (CBIR) is an essential part of computer vi...

Scene Recognition by Combining Local and Global Image Descriptors

Object recognition is an important problem in computer vision, having di...

Combined Descriptors in Spatial Pyramid Domain for Image Classification

Recently spatial pyramid matching (SPM) with scale invariant feature tra...

Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence

We present a novel descriptor, called deep self-convolutional activation...

1 Introduction

Breast cancer is the most common and leading cause of cancer death in women [1]. For breast cancer diagnosis, mammography, which uses X-rays to examine the human breast, is usually used for early diagnosis. However, even for skilled radiologists, the mammogram diagnosis between benign and malignant is difficult due to their similarity in the target shape and texture. In order to reduce the human bias as well as to automate the process, on-going effort has been paid to developing Computer-Aided Diagnosis (CAD) systems for mammogram classification . Typically, these systems rely on extraction of the texture and the shape information from the radio images [2], [3]. The mammogram classification performance depends heavily on precise segmentations of the target tumors (i.e., complex lesion boundary of the tumors) for reliable features. To avoid this dependency, the texture information obtained from local descriptors such as the LBP [4], the Local Binary Convolution (LBC) [5] and the HOG [6] are adopted [7], [8].

Our motivation for mammogram classification consists of the following: (i) The shape based feature extraction can be significantly affected by the irregular lesion boundaries. (ii) The texture based feature extraction such as LBP, HOG and LBC typically use the convolution based iterative scanning process that is time-consuming and often produces a large feature dimension. (iii) To utilize the diversity of different local descriptors for performance improvement.

The main contributions of this work are as follows: (i) Matrix formulations of two descriptors, namely the LBP and the HOG, to avoid the iterative convolution process in local computation and to get away from the high dimensional feature size. (ii) An integrated formulation of the LBP and the HOG under linear matrix products.

The remainder of the paper is organized as follows: Section 2

includes a brief review of the LBC descriptor, the DMP (Difference Matrix Projection) method and the total-error-rate based classifier. In Section

3, we propose two convolution-free descriptors in matrix formulations. Then we construct an integrated matrix formulation of these two descriptors. Section 4 presents our experiments, observations and evaluations of the proposed method using a public mammogram database. Concluding remarks are given in Section 5.

2 Preliminaries

2.1 Local Binary Convolution network (LBC)

The Local Binary Pattern (LBP) [4] is a simple and popular descriptor adopted in many applications (e.g., face and palm print recognitions [9, 10, 11]). The LBP scans each central pixel of an image and its local neighborhood pixels (

) within an odd size window determined by

(e.g., indicates a window and indicates a window). The computed output of the LBP can be expressed as , where and are respectively the intensity values of the center pixel and the neighborhood pixels. is the thresholding operation given by . Frequently, for images with high enough resolution, the, and are set at and .

Based on the LBP, a Local Binary Convolution network (LBC) was proposed in [5]

to encode the LBP efficiently utilizing the sparse convolutional filters. The LBC was developed as an alternative to the standard Convolutional Neural Networks (CNN) layer to reduce its complexity by reducing the number of learnable parameters. For a

window size, the LBC uses a weighted sum of eight sparse convolutional filters to produce the binary map. The original LBP is consequently reformulated in [5] as the basis of LBC as follows:


where is the convolution operation, and is the sparse convolutional filter with two non-zero values

which convolves with the vectorized input image

, is the Heaviside step function. This function

can be replaced by a Sigmoid or a ReLU activation function for differentiability.

contains predefined weight values. is the number of neighbor pixels (e.g., with window).

2.2 Difference Matrix Projection (DMP)

The Difference Matrix Projection (DMP) was proposed for pedestrian detection in [12]. Compared with the pixel-wise calculations of the Histogram of Oriented Gradients (HOG) [6] based on the first order gradients, the DMP realized an approximated HOG based on linear matrix products utilizing both the first and the second order gradients with pre-calculated projection matrices. The DMP process uses a cell-based pooling to substitute the histogram construction.

The DMP process can be written as:




are the predefined projection matrices for cell-based non-overlapping pooling, and is the cell size. Based on the first and the second order gradients from four orientations (, , and ), the gradient can be expressed as:


where to are the first-order gradients and to are the second-order gradients. is the input image. The predefined horizontal and vertical shifting matrices are given respectively by



is the identity matrix.

indicates the number of shifting pixels which determine the first and the second order gradients when and respectively. Similar to HOG in [6], the block normalization is subsequently performed on the cell-based pooling to obtain the final DMP features.

2.3 Total-Error-Rate (TER) Minimization

The Total-Error-Rate (TER) based classification [13]

utilized the sum of the type I and type II errors. The type I error (also known as False Positive Rate (FPR)) is the ratio of falsely recognized positive samples over the negative sample size given by

. The type II error (also known as False Negative Rate (FNR)) is the ratio of falsely recognized negative samples over the positive sample size given by . Then, TER can be written as . For a classifier which is linear in its parameters, the TER parameters can be optimally determined by


where , and are respectively the transformed samples of each category, is the total number of samples and is a class-specific weight matrix where , . is the learning target vector, with and for a given threshold and offset . is a vector of element ones. is the regularization factor with being the identity matrix that matches the dimension of .

3 A Convolution-Free LBP-HOG Descriptor

3.1 Overview

In this section, we propose a Convolution-Free LBP-HOG descriptor (CF-LBP-HOG) in matrix form. Specifically, we propose a Convolution-Free LBP descriptor (CF-LBP) in matrix form in the first step. This is followed by a Convolution-Free HOG descriptor (CF-HOG). The proposed CF-LBP and CF-HOG are then integrated into a single matrix product form.

3.2 Convolution-Free LBP in Matrix Form

The original LBP and the LBC iteratively compute the pixel differences in eight directions at angle difference ( to ). The proposed CF-LBP can be formulated as the weighted sum of eight directional difference matrices as follows:




with and being the predefined shifting matrices from equation (5).

3.3 Convolution-Free HOG in Matrix Form

In [12], the DMP method utilized the window based iterative scanning process for block normalization based on the -norm. The proposed CF-HOG as an approximated HOG is based on the norm block normalization in matrix form including overlapping. A cell-based overlapping pooling is also proposed to obtain the local connection between the cell groups. The cell-based overlapping pooling can be written as follows:




and is the cell-based non-overlapping pooling from equation (2) using a cell size of in which and are the predefined projection matrices for the cell-based overlapping pooling from equation (10) and (11) using a cell size of and an overlapping size of .

In [6], the normalization is applied to each block which consisted of a cell group. The block-based overlapping normalization in matrix form can be written as follows:


where denotes the Hadamard elementwise operation. and are the predefined projection matrices for the block-based overlapping normalization. is the sum of the squared elements of each block based on the pooling technique. and are for upsampling to retain the original matrix size. The superscript is the elementwise inverse of the squared root.

3.4 Integrating LBP and HOG into One Matrix Form

Based on the CF-LBP and CF-HOG, the proposed CF-LBP-HOG in matrix form can be written as follows:


where , , , , is the input matrix utilized by in equation (7) and is obtained by using equation (4) with . For the final CF-LBP-HOG features, each are concatenated together into one feature vector.

3.5 A Case Study of The LBP Based Descriptors

In this study, we compare the proposed CF-LBP with the original LBP and with the LBC using a mammogram image as the input image. Based on a typical LBP setting at and , the LBP based descriptors, namely, LBP, LBC and CF-LBP, are performed to obtain the LBP based texture image. Fig. 1 shows the same output values within the borders from each descriptor whereas the borders of each descriptor have different output values due to different techniques to perform the LBP.

Figure 1: An illustrative comparison among the original LBP, the LBC and the proposed CF-LBP using and .
(a) Comparison of test accuracies at
(b) Comparison of test accuracies at
(c) Comparison of test accuracies at
Figure 2: Comparison of HOG, DMP and CF-HOG in terms of classification test accuracies (%) on CBIS-DDSM database.

4 Experiments

In this section, we evaluate the proposed descriptor for mammogram classification. The experimental goals are as follows: 1) Observing the effect of overlapping pooling among the HOG, the DMP and the proposed CF-HOG; 2) Performance comparison of the proposed CF-LBP-HOG with state-of-the-arts descriptors.

4.1 Database and Experimental Setup

The most commonly used database in mammography is the Digital Database of Screening Mammography (DDSM) [14]. Recently, in [15], a Curated Breast Imaging Subset of the DDSM (CBIS-DDSM) has been released in view of the segmentation difficulty for a standarized evaluation. In this experiment, we used the CBIS-DDSM database. The database includes ROI images which are categorized into two classes (malignant, benign) for patients and the images are resized to a resolution. By following the data split setting in [15], a training set ( images) and a test set ( images) are obtained.

For state-of-the-art descriptors, the original LBP [4], the LBC [5], the HOG [6] and the DMP [12] are implemented for comparison. The LBP-HOG is also implemented based on a cascade of the LBP image and the HOG feature representation. For a state-of-the-art mammogram classification system, the VGGNet [16] is additionally implemented. For the parameter settings of each descriptor, we followed the general settings [4], [5], [6], [12] for LBP, HOG, DMP, LBP-HOG and the proposed CF-LBP, CF-HOG and CF-LBP-HOG: , , Heaviside step function, , , and . The histogram bin sizes of LBP and HOG are respectively fixed at and

. The TER and SVM classifiers are utilized with a radial basis function (RBF). The regularization parameter

of the TER is fixed at . According to [16], the parameters settings of the VGGNet is set.

4.2 Observing the effect of the overlapping pooling among the HOG based descriptors

To compare the proposed CF-HOG descriptor to the HOG and DMP descriptors, classification test accuracies at different cell and block sizes ( and ) were acquired utilizing CBIS-DDSM database. Fig. 2 shows the effect of the proposed overlapping pooling at different cell size in CF-HOG compared to HOG and DMP where there is no overlapping pooling. The accuracy performances are observed according to increasing the non-overlapping pooling size . According to the increment, the performance of the CF-HOG is increasing while that of HOG and DMP are unstable. The proposed CF-HOG outperformed the other state-of-the-art methods when is and . The best performance is observed in CF-HOG when the overlapping pooling is included.

VGGNet 63.35
SVM 60.79 60.80 60.51 63.49 62.36
TER 56.25 59.52 60.23 62.07 64.35
Table 1: Comparison of Classification Test Accuracies (%)
Extraction Time
VGGNet 150,528 - 174,460 209.88
LBP 19,116 53.53 24.40 0.0069
HOG 10,404 42.50 14.59 0.0038
DMP 9,248 62.00 12.97 0.0034
LBP-HOG 10,404 43.21 14.25 0.0038
CF-LBP-HOG 1,800 31.13 04.00 0.0008
Table 2: Comparison of CPU Processing Time in Seconds

4.3 Performance Comparison and Summary

In terms of classification performance, Table 1 shows the classification accuracies for the compared descriptors, namely LBP, HOG, DMP, LBP-HOG and CF-LBP-HOG. The proposed CF-LBP-HOG based on the TER classifier showed the best performance compared to the state-of-the-arts while the LBP-HOG based on the SVM classifier and the VGGNet respectively showed the second and the third bests.

In terms of computational performance, Table 2 shows the CPU processing time in seconds. The averaged CPU times are reported over 10 runs. The proposed CF-LBP-HOG based on the TER classifier showed the best CPU times in the training and test phases due to the predefined projection matrices under global computation form. In addition, the CF-LBP-HOG produced the smallest feature dimension which is 5 times less than the other descriptors.

In summary, we have shown that 1) the overlapping pooling step of the proposed CF-HOG is effective compared with the HOG and the DMP, 2) the proposed CF-LBP-HOG achieved better performance with a small feature dimension than that of the other state-of-the-art methods in terms of classification accuracy and CPU time.

5 Conclusion

Different from the convolution based LBP, we have presented a convolution-free LBP (CF-LBP) in matrix form. In addition, we have shown a convolution-free HOG based on Difference Matrix Projection (DMP). The integrated form of these two proposed descriptors, CF-LBP-HOG, was then proposed in a matrix formulation. The proposed descriptors were evaluated using the CBIS-DDSM database for mammogram classification where the results show promising performance comparing with state-of-the-art descriptors.


  • [1] Freddie Bray, Jacques Ferlay, Isabelle Soerjomataram, Rebecca L. Siegel, Lindsey A. Torre, and Ahmedin Jemal,

    “Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries,”

    CA: A Cancer Journal For Clinicians, vol. 68, no. 6, pp. 394–424, 2018.
  • [2] Hamid Soltanian-Zadeh, Siamak Pourabdollah-Nezhad, and Farshid Rafiee Rad, “Shape-based and texture-based feature extraction for classification of microcalcifications in mammograms,” in Medical Imaging: Image Processing. International Society for Optics and Photonics, 2001, vol. 4322, pp. 301–311.
  • [3] Alan Joseph Bekker, Moran Shalhon, Hayit Greenspan, and Jacob Goldberger, “Multi-view probabilistic classification of breast microcalcifications,” IEEE Transactions on Medical Imaging, vol. 35, no. 2, pp. 645–653, 2016.
  • [4] Timo Ojala, Matti Pietikainen, and David Harwood, “Performance evaluation of texture measures with classification based on kullback discrimination of distributions,” in

    International Conference on Pattern Recognition (ICPR)

    . IEEE, 1994, vol. 1, pp. 582–585.
  • [5] Felix Juefei-Xu, Vishnu Naresh Boddeti, and Marios Savvides, “Local binary convolutional neural networks,” in

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    . IEEE, 2017, vol. 1.
  • [6] Navneet Dalal and Bill Triggs, “Histograms of oriented gradients for human detection,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2005, vol. 1, pp. 886–893.
  • [7] Arnau Oliver, Xavier Lladó, Robert Marti, Jordi Freixenet, and Reyer Zwiggelaar, “Classifying mammograms using texture information,” in Medical Image Understanding and Analysis. Citeseer, 2007, vol. 223.
  • [8] Aditya A. Shastri, Deepti Tamrakar, and Kapil Ahuja, “Density-wise two stage mammogram classification using texture exploiting descriptors,” Expert Systems with Applications, vol. 99, pp. 71–82, 2018.
  • [9] Timo Ahonen, Abdenour Hadid, and Matti Pietikainen,

    “Face description with local binary patterns: Application to face recognition,”

    IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006.
  • [10] Alireza Sepas-Moghaddam, Paulo Lobato Correia, and Fernando Pereira, “Light field local binary patterns description for face recognition,” in IEEE International Conference on Image Processing (ICIP). IEEE, 2017, pp. 3815–3819.
  • [11] Yufei Han, Zhenan Sun, and Tieniu Tan, “Palmprint recognition using coarse-to-fine statistical image representation,” in IEEE International Conference on Image Processing (ICIP). IEEE, 2009, pp. 1969–1972.
  • [12] Xing Liu and Kar-Ann Toh, “A novel set of pixel difference-based features for pedestrian detection,” in International Conference on Identity, Security, and Behavior Analysis (ISBA). IEEE, 2018, pp. 1–6.
  • [13] Kar-Ann Toh, “Deterministic neural classification,” Neural Computation, vol. 20, no. 6, pp. 1565–1595, 2008.
  • [14] Michael Heath, Kevin Bowyer, Daniel Kopans, Richard Moore, and W. Philip Kegelmeyer, “The digital database for screening mammography,” in International Workshop on Digital Mammography. Medical Physics Publishing, 2000, pp. 212–218.
  • [15] Rebecca Sawyer Lee, Francisco Gimenez, Assaf Hoogi, Kanae Kawai Miyake, Mia Gorovoy, and Daniel L. Rubin, “A curated mammography data set for use in computer-aided detection and diagnosis research,” Scientific Data, vol. 4, pp. 170–177, 2017.
  • [16] Pengcheng Xi, Chang Shu, and Rafik Goubran, “Abnormality detection in mammography using deep convolutional neural networks,” in IEEE International Symposium on Medical Measurements and Applications (MeMeA). IEEE, 2018.