1 Introduction
Saliency detection aims to effectively highlight the most important pixels in an image. It helps to reduce computing costs and has widely been used in various computer vision applications, such as image segmentation
[1][2], object detection [3, 4], object recognition [5], image adaptation [6], and video segmentation [7, 8]. Saliency detection could be summarized in three methods: bottomup methods [9, 10, 11], topdown methods [12, 13] and mixed methods [14, 15, 16]. The topdown methods are driven by tasks and could be used in object detection tasks. The authors in [17] proposed a topdown method that jointly learns a conditional random field and a discriminative dictionary. Topdown methods could be applied to address complex and special tasks but they lack versatility. The bottomup methods are driven by data, such as color, light, texture and other basic features. Itti et al [18] proposed a saliency method by using these basic features. It could be effectively used for realtime systems. The mixed methods are considered both bottomup and topdown methods.In this paper, we focus on the bottomup methods, the proposed method is based on the properties of Markov model, there are many works based on Markov model, such as
[19, 20]. Traditional saliency detection via Markov chain [21] is based on Marov model as well, but it only consider boundaries nodes. However, in addition to boundaries cues, background prior and foreground prior cues play a complementary role to enhance saliency detection. We consider four boundaries information and the foreground prior saliency object, using absorbing Markov chain, namely, both boundary absorbing and foreground prior are considered to get background and foreground possibility. In addition, we further optimize our model by fusing these two possibilities, and exploite multiscale processing. Fig.1 demonstrates and compares the results of our proposed method with the traditional saliency detection absorbing Markov chain (MC) method [21], where the outperformance of our method is evident.2 Related works
There are existing many studies on saliency detection in the past decades according to the resent surveys [22, 23, 24]. And the proposed model ia based on absorbing Markov chain belonging to bottomup method, therefore, in this section, we mainly focus on the traditional models and other newly models.
The traditional models which belong to bottomup methods, have the features of unconscious, fast, data driven, lowlevel feature driven, which means without any prior knowledge, bottomup saliency detection can achieve the goal of finding the important regions in a image. The earliest researchers Itti and Koch model [18, 25], which compute saliency maps though lowlevel features, such as texture, orientation, intensity, and color contrast. From then on, multitudinous traditional saliency models have been appeared and acquired outperformances. Some methods based on pixel [26, 27, 28], Some methods based on superpixel [29, 30], and others based on multiscale [31, 32, 33]. Jiang et al [21]
propose saliency detection model by random walk via absorbing Markov Chain where absorbing nodes are duplicated from the four boundaries, and compute absorbing time from the transient nodes to absorbing nodes, obtain the final saliency maps. Considering the importance of the transition probability matrix, Zhang et al
[34] based on their aforementioned work propose a learnt transition probability matrix to improve the perfermance. There are some other work based on Markov chain. Zhang et al [35] propose an approach to detection salient objects by exporing both patchlevel and objectlevel cues via absorbing Markov chain. Zhang et al [36] present a datadriven salient region detection model based on absorbing Markov chain via multifeature. Zhu et al [37] integrate boundary connectivity based on geodesic distances into a cost function to get the final optimized saliency map. Li et al [38]propose a saliency optimization scheme by considering both the foreground appearance and background prior. Resent, due to the great developing of deep learning, there are numerous saliency detection models
[11, 15, 39, 40] based on deep learning, which obtain outperformance than the traditional methods. However, deep learning based methods need much dates to train, costing much time on computation, which makes the proceeding of saliency detection much complex than before. In this work, based on the traditional method, we introduce bidirectional absorbing Markov chain to this kinds of work to get excellent performance in saliency detection.3 Fundamentals of absorbing Markov chain
In absorbing Markov chain, the transition matrix is primitive [41], by definition, state is absorbing when , and for all . If the Markov chain satisfies the following two conditions, it means there is at least one or more absorbing states in the Markov chain. In every state, it is possible to go to an absorbing state in a finite number of steps(not necessarily in one step), then we call it absorbing Markov chain. In an absorbing Markov chain, if a state is not a absorbing state, it is called transient state.
An absorbing chain has absorbing states and transient states, the transfer matrix can be written as:
(1) 
where is a nbyn matrix, giving transient probabilities between any transient states, is a nonzero nbym matrix giving these probabilities from transient state to any absorbing state,
is a mbyn zero matrix and
is the mbym identity matrix.
For an absorbing chain , all the transient states can achieve absorbing states in one or more steps, we can write the expected number of times (which means the transient state moves from state to the state), its standard form is written as:
(2) 
namely, the matrix
with invertible matrix, where
denotes the average transfer times between transient state to transient state . Supposing , the absorbed time for each transient state can be expressed as:(3) 
4 The proposed approach
To obtain more robust and accurate saliency maps, we propose a method via bidirectional absorbing Markov chain. This section explains the procedure to find the saliency area in an image in two orientations. Simple linear iterative clustering(SLIC) algorithm [42] has been used to get the superpixels. The pipeline is explained below:
4.1 Graph construction
The SLIC algorithm is used to split the image into different pitches of superpixels. Afterwards, two kinds of graphs and are constructed, see Figure 3 for detail,
4.2 Graph construction
where represents the graph of boundary absorbing process and represents the graph of foreground prior absorbing process. In each of the graphs, represent the graph nodes and represent the edges between any nodes in the graphs. For the process of boundary absorbing, superpixels around the four boundaries as the virtual nodes are duplicated. For the process of foreground prior absorbing, superpixels from the regions (calculated by the foreground prior) are duplicated. There are two kinds of nodes in both graphs, transient nodes (superpixels) and absorbing nodes (duplicated nodes). The nodes in these two graphs constitute following three properties: (1) The nodes (including transient or absorbing) are associated with each other when superpixels in the image are adjacent nodes or have the same neighbors. And also boundary nodes (superpixels on the boundary of image) are fully connected with each other to reduce the geodesic distance between similar superpixels. (2) Any pair of absorbing nodes (which are duplicated from the boundaries or foreground nodes) are not connected (3) The nodes, which are duplicated from the four boundaries or foreground prior nodes, are also connected with original duplicated nodes. In this paper, the weight of the edges is defined as
(4) 
where
is the constant parameter to adjust the strength of the weights in CIELAB color space. Then we can get the affinity matrix
(5) 
where is a nodes set, in which the nodes are all connected to nodes . The diagonal matrix is given as: , and the obtained transient matrix is calculated as:
4.3 Saliency detection model
Following the aforementioned procedures, the initial image is transformed into superpixels, now two kinds of absorbing nodes for saliency detection are required. Firstly, we choose boundary nodes and foreground prior nodes to duplicate as absorbing nodes and obtain the absorbed times of transient nodes as foreground possibility and background possibility. Secondly, we use a cost function to optimize two possibility results together and obtain saliency results of all transient nodes.
4.3.1 Absorb Markov chain via boundary nodes
In normal conditions, four boundaries of an image rarely have salient objects. Therefore, boundary nodes are assumed as background, and four boundaries nodes set are duplicated as absorbing nodes set , . The graph is constructed and absorbed time is calculated via Eq.3. Finally, foreground possibility of transient nodes is obtained, and
denotes the normalizing the absorbed time vector.
4.3.2 Absorb Markov chain via foreground prior nodes
We use boundary connectivity to get the foreground prior without using the downtop method [37].
(6) 
where and denote the CIELAB color feature distance and spatial distance respectively between superpixel and , the boundary connectivity (BC) of superpixel is defined as in Fig. 4, , . denotes the boundary area of image and is the similarity between nodes and . is the number of superpixels. Afterwards, nodes () with high level values are selected to get a set , which are duplicated as absorbing nodes set , . The graph is constructed and absorbed time is calculated using Eq.3. Finally, the background possibility of transient nodes is obtained, where denotes the absorbed time vector normalization.
4.4 Saliency Optimization
In order to combine different cues, this paper has used the optimization model presented in [37], which fused background possibility and foreground possibility for final saliency map. It is defined as
(7) 
where the first term defines superpixel with large background probability to obtain a small value (close to 0). The second term encourages a superpixel with large foreground probability to obtain a large value (close to 1). The third term defines the smoothness to acquire continuous saliency values.
In this work, the used superpixel numbers are 200, 250, 300 in the superpixel element, and the final saliency map is given as: at each scale, where . The algorithm of our proposed method is summarized in Algorithm 1.
5 Experiments
The proposed method is evaluated on four benchmark datasets ASD [43], CSSD [31], ECSSD [31] and SED [44]. ASD dataset is a subset of the MSRA dataset, which contains 1000 images with accurate humanlabeled ground truth. CSSD dataset, namely complex scene saliency detection contains 200 complex images. ECSSD dataset, an extension of CSSD dataset contains 1000 images and has accurate humanlabeled ground truth. SED dataset has two parts, SED1 and SED2, images in SED1 contains one object, and images in SED2 contains two objects, in total they contain 200 images. We compare our model with 17 different stateoftheart saliency detection algorithms: CA [45], FT [43], SEG [46], BM [47], SWD [48], SF [49], GCHC [50], LMLC [51], HS [31], PCA [30], DSR [52], MC [21], MR [10], MS [53], RBD [37], RR [54], MST [55]. The tuning parameters in the proposed algorithm is the edge weight that controls the strength of weight between a pair of nodes. In the following results of the experiments, we show the evaluation of our proposed saliency models based on the aforementioned datasets comparing with the best works. In addition, we also give some limitation about our model and analysis the reason.
5.1 Evaluation of the proposed model
The precisionrecall curves and Fmeasure are used as performance metrics. The precision is defined as the ratio of salient pixels correctly assigned to all the pixels of extracted regions. The recall is defined as the ratio of detected salient pixels to the groundtruth number. Which can be fomulated as,
(8) 
where , and represent the true positive, false positive and false negative, respectively. A PR curve is obtained by the threshold sliding from 0 to 255 to get the difference between the saliency map (which is calculated) and ground truth(which is labeled manually). Fmeasure is calculated by the weighted average between the precision values and recall values, which can be regarded as overall performance measurement, given as:
(9) 
we set to stress precision more than recall. PRcurves and the Fmeasure curves are shown in Figure 5  8, where the outperformance of our proposed method as compared to 17 stateoftheart methods is evident. Fig.9 presets visual comparisons selected from four datasets. It can be seen that the proposed method achieved best saliency results as compared to the stateoftheart methods.
5.2 Failure cases analysis
In this work, the idea of bidirectional absorbing Markov chain is first proposed. Although the proposed method is effective for most images on the four datasets. However, if the appearances of four boundaries and the foreground prior are similar to each other, the performance is not obviously, which is shown in Figure 10.
6 Conclusion
In this paper, a bidirectional absorbing Markov chain based saliency detection method is proposed considering both boundary information and foreground prior cues. A novel optimization model is developed to combine both background and foreground possibilities, acquired through bidirectional absorbing Markov chain. The proposed approach outperformed 17 different stateoftheart methods over four benchmark datasets, which demonstrate the superiority of our proposed approach. In future, we intend to apply our proposed saliency detection algorithm to problems such as multipose lipreading and audiovisual speech recognition.
7 Acknowledgments
This work was supported by China Scholarship Council, the National Natural Science Foundation of China (No.913203002), the Pilot Project of Chinese Academy of Sciences (No.XDA08040109). Prof. Amir Hussain and Dr. Ahsan Adeel were supported by the UK Engineering and Physical Sciences Research Council (EPSRC) grant No.EP/M026981/1.
8 Reference
References
 [1] P. Arbelaez, M. Maire, C. Fowlkes, J. Malik, Contour detection and hierarchical image segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 33 (5) (2011) 898–916.
 [2] X. Yang, X. Qian, Y. Xue, Scalable mobile image retrieval by exploring contextual saliency, IEEE Trans. on Image Processing 24 (6) (2015) 1709–1721.
 [3] K.Y. Chang, T.L. Liu, H.T. Chen, S.H. Lai, Fusing generic objectness and visual saliency for salient object detection, in: Proc. of IEEE Int. Conf. on Computer Vision (ICCV), IEEE, 2011, pp. 914–921.

[4]
F. Gao, Y. Zhang, J. Wang, J. Sun, E. Yang, A. Hussain, Visual attention model based vehicle target detection in synthetic aperture radar images: a novel approach, Cognitive Computation 7 (4) (2015) 434–444.
 [5] Z. Ren, S. Gao, L.T. Chia, I. W.H. Tsang, Regionbased saliency detection and its application in object recognition, IEEE Trans. on Circuits and Systems for Video Technology 24 (5) (2014) 769–779.
 [6] J. Sun, J. Xie, J. Liu, T. Sikora, Image adaptation and dynamic browsing based on twolayer saliency combination, IEEE Trans. on Broadcasting 59 (4) (2013) 602–613.
 [7] W. Wang, J. Shen, R. Yang, F. Porikli, Saliencyaware video object segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 40 (1) (2018) 20–33.
 [8] Z. Tu, A. Abel, L. Zhang, B. Luo, A. Hussain, A new spatiotemporal saliencybased video object segmentation, Cognitive Computation 8 (4) (2016) 629–647.
 [9] N. Riche, M. Mancas, B. Gosselin, T. Dutoit, Rare: A new bottomup saliency model, in: Proc. of the 19th IEEE Int. Conf. on Image Processing (ICIP), IEEE, 2012, pp. 641–644.

[10]
C. Yang, L. Zhang, H. Lu, X. Ruan, M.H. Yang, Saliency detection via graphbased manifold ranking, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2013, pp. 3166–3173.
 [11] R. Zhao, W. Ouyang, H. Li, X. Wang, Saliency detection by multicontext deep learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1265–1274.
 [12] J. Yang, M.H. Yang, Topdown visual saliency via joint crf and dictionary learning, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 39 (3) (2017) 576–588.

[13]
H. Cholakkal, J. Johnson, D. Rajan, Backtracking scspm image classifier for weakly supervised topdown saliency, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016, pp. 5278–5287.
 [14] A. Borji, D. N. Sihite, L. Itti, Probabilistic learning of taskspecific visual attention, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2012, pp. 470–477.
 [15] Z. Wang, J. Ren, D. Zhang, M. Sun, J. Jiang, A deeplearning based feature hybrid framework for spatiotemporal saliency detection inside videos, Neurocomputing 287 (2018) 68–83.
 [16] Y. Yan, J. Ren, G. Sun, H. Zhao, J. Han, X. Li, S. Marshall, J. Zhan, Unsupervised image saliency detection with gestaltlaws guided optimization and visual attention based refinement, Pattern Recognition 79 (2018) 65–78.
 [17] J. Yang, M.H. Yang, Topdown visual saliency via joint crf and dictionary learning, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2012, pp. 2296–2303.
 [18] L. Itti, C. Koch, E. Niebur, A model of saliencybased visual attention for rapid scene analysis, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 20 (11) (1998) 1254–1259.

[19]
J. H. AlKhateeb, O. Pauplin, J. Ren, J. Jiang, Performance of hidden markov model and dynamic bayesian network classifiers on handwritten arabic word recognition, Knowledgebased Systems 24 (5) (2011) 680–688.
 [20] J. H. AlKhateeb, J. Ren, J. Jiang, H. AlMuhtaseb, Offline handwritten arabic cursive text recognition using hidden markov models and reranking, Pattern Recognition Letters 32 (8) (2011) 1081–1088.
 [21] B. Jiang, L. Zhang, H. Lu, C. Yang, M.H. Yang, Saliency detection via absorbing markov chain, in: Proc. of IEEE Int. Conf. on Computer Vision (ICCV), IEEE, 2013, pp. 1665–1672.
 [22] A. Borji, L. Itti, Stateoftheart in visual attention modeling, IEEE transactions on pattern analysis and machine intelligence 35 (1) (2013) 185–207.
 [23] A. Borji, M.M. Cheng, H. Jiang, J. Li, Salient object detection: A benchmark, IEEE Transactions on Image Processing 24 (12) (2015) 5706–5722.
 [24] D. Zhang, H. Fu, J. Han, A. Borji, X. Li, A review of cosaliency detection algorithms: Fundamentals, applications, and challenges, ACM Transactions on Intelligent Systems and Technology (TIST) 9 (4) (2018) 38.
 [25] L. Itti, C. Koch, Computational modelling of visual attention, Nature reviews neuroscience 2 (3) (2001) 194.
 [26] Y. Zhai, M. Shah, Visual attention detection in video sequences using spatiotemporal cues, in: Proceedings of the 14th ACM international conference on Multimedia, ACM, 2006, pp. 815–824.
 [27] K. Shi, K. Wang, J. Lu, L. Lin, Pisa: Pixelwise image saliency by aggregating complementary appearance contrast measures with spatial priors, in: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEE, 2013, pp. 2115–2122.
 [28] M.M. Cheng, J. Warrell, W.Y. Lin, S. Zheng, V. Vineet, N. Crook, Efficient salient region detection with soft image abstraction, in: Computer Vision (ICCV), 2013 IEEE International Conference on, IEEE, 2013, pp. 1529–1536.
 [29] Y. Wei, F. Wen, W. Zhu, J. Sun, Geodesic saliency using background priors, in: European conference on computer vision, Springer, 2012, pp. 29–42.
 [30] R. Margolin, A. Tal, L. ZelnikManor, What makes a patch distinct?, in: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEE, 2013, pp. 1139–1146.
 [31] Q. Yan, L. Xu, J. Shi, J. Jia, Hierarchical saliency detection, in: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEE, 2013, pp. 1155–1162.
 [32] H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, Salient object detection: A discriminative regional feature integration approach, in: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEE, 2013, pp. 2083–2090.
 [33] L. Zhang, Y. Zhang, H. Yan, Y. Gao, W. Wei, Salient object detection in hyperspectral imagery using multiscale spectralspatial gradient, Neurocomputing 291 (2018) 215–225.
 [34] L. Zhang, J. Ai, B. Jiang, H. Lu, X. Li, Saliency detection via absorbing markov chain with learnt transition probability, IEEE Transactions on Image Processing 27 (2) (2018) 987–998.
 [35] Q. Zhang, D. Luo, W. Li, Y. Shi, J. Lin, Twostage absorbing markov chain for salient object detection, in: Image Processing (ICIP), 2017 IEEE International Conference on, IEEE, 2017, pp. 895–899.
 [36] W. Zhang, Q. Xiong, W. Shi, S. Chen, Region saliency detection via multifeature on absorbing markov chain, The Visual Computer 32 (3) (2016) 275–287.
 [37] W. Zhu, S. Liang, Y. Wei, J. Sun, Saliency optimization from robust background detection, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2014, pp. 2814–2821.
 [38] L. Li, F. Zhou, Y. Zheng, X. Bai, Saliency detection based on foreground appearance and backgroundprior, Neurocomputing 301 (2018) 46–61.
 [39] L. Wang, L. Wang, H. Lu, P. Zhang, X. Ruan, Saliency detection with recurrent fully convolutional networks, in: European Conference on Computer Vision, Springer, 2016, pp. 825–841.
 [40] G. Lee, Y.W. Tai, J. Kim, Eldnet: An efficient deep learning architecture for accurate saliency detection, IEEE transactions on pattern analysis and machine intelligence 40 (7) (2018) 1599–1610.
 [41] M. Charles, J. Grinstead, L. Snell, Introduction to probability, American Mathematical Society.
 [42] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Süsstrunk, Slic superpixels compared to stateoftheart superpixel methods, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 34 (11) (2012) 2274–2282.
 [43] R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, Frequencytuned salient region detection, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2009, pp. 1597–1604.
 [44] S. Alpert, M. Galun, A. Brandt, R. Basri, Image segmentation by probabilistic bottomup aggregation and cue integration, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 34 (2) (2012) 315–327.
 [45] S. Goferman, L. ZelnikManor, A. Tal, Contextaware saliency detection, IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI) 34 (10) (2012) 1915–1926.
 [46] E. Rahtu, J. Kannala, M. Salo, J. Heikkilä, Segmenting salient objects from images and videos, in: Proc. of European Conf. on Computer Vision (ECCV), Springer, 2010, pp. 366–379.
 [47] Y. Xie, H. Lu, Visual saliency detection based on bayesian model, in: Proc. of the 18th IEEE Int. Conf. on Image Processing (ICIP), IEEE, 2011, pp. 645–648.
 [48] L. Duan, C. Wu, J. Miao, L. Qing, Y. Fu, Visual saliency detection by spatially weighted dissimilarity, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2011, pp. 473–480.
 [49] F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, Saliency filters: Contrast based filtering for salient region detection, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2012, pp. 733–740.
 [50] C. Yang, L. Zhang, H. Lu, Graphregularized saliency detection with convexhullbased center prior, IEEE Signal Processing Letters 20 (7) (2013) 637–640.
 [51] Y. Xie, H. Lu, M.H. Yang, Bayesian saliency via low and mid level cues, IEEE Trans. on Image Processing 22 (5) (2013) 1689–1698.
 [52] X. Li, H. Lu, L. Zhang, X. Ruan, M.H. Yang, Saliency detection via dense and sparse reconstruction, in: Proc. of IEEE Int. Conf. on Computer Vision (ICCV), IEEE, 2013, pp. 2976–2983.
 [53] N. Tong, H. Lu, L. Zhang, X. Ruan, Saliency detection with multiscale superpixels, IEEE Signal Processing Letters 21 (9) (2014) 1035–1039.
 [54] C. Li, Y. Yuan, W. Cai, Y. Xia, D. D. Feng, et al., Robust saliency detection via regularized random walks ranking., in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015, pp. 2710–2717.
 [55] W.C. Tu, S. He, Q. Yang, S.Y. Chien, Realtime salient object detection with a minimum spanning tree, in: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016, pp. 2334–2342.
Comments
There are no comments yet.