An ILP Solver for Multi-label MRFS with Connectivity Constraints

12/16/2017
by   Ruobing Shen, et al.
University of Heidelberg
0

Integer Linear Programming (ILP) formulations of Markov random fields (MRFs) models with global connectivity priors were investigated previously in computer vision, e.g., globalinter,globalconn. In these works, only Linear Programming (LP) relaxations globalinter,globalconn or simplified versions graphcutbase of the problem were solved. This paper investigates the ILP of multi-label MRF with exact connectivity priors via a branch-and-cut method, which provably finds globally optimal solutions. The method enforces connectivity priors iteratively by a cutting plane method, and provides feasible solutions with a guarantee on sub-optimality even if we terminate it earlier. The proposed ILP can also be applied as a post-processing method on top of any existing multi-label segmentation approach. We demonstrate the power and usefulness of our model by several experiments on the BSDS500 image dataset, as well as on medical images with trained probability maps.

READ FULL TEXT VIEW PDF

Authors

page 5

page 6

page 7

page 8

11/24/2014

Iteratively Reweighted Graph Cut for Multi-label MRFs with Non-convex Priors

While widely acknowledged as highly effective in computer vision, multi-...
09/06/2019

Strong Bounds for Resource Constrained Project Scheduling: Preprocessing and Cutting Planes

Resource Constrained Project Scheduling Problems (RCPSPs) without preemp...
08/26/2019

Constraint Generation Algorithm for the Minimum Connectivity Inference Problem

Given a hypergraph H, the Minimum Connectivity Inference problem asks fo...
06/05/2019

Anatomical Priors for Image Segmentation via Post-Processing with Denoising Autoencoders

Deep convolutional neural networks (CNN) proved to be highly accurate to...
06/18/2020

An Integer Linear Programming Framework for Mining Constraints from Data

Various structured output prediction problems (e.g., sequential tagging)...
06/24/2020

Post-DAE: Anatomically Plausible Segmentation via Post-Processing with Denoising Autoencoders

We introduce Post-DAE, a post-processing method based on denoising autoe...
08/14/2013

Compact Relaxations for MAP Inference in Pairwise MRFs with Piecewise Linear Priors

Label assignment problems with large state spaces are important tasks es...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Most early vision problems can be formulated using Markov Random Fields (MRFs), hence its solution algorithms are of pivotal importance in computer vision. The MAP-MRF (maximizing a posteriori in an MRF) has proven to be successful for many computer vision problems such as image segmentation, denoising and stereo, among others. We refer to [4, 5, 6] for an overview of MRF optimization techniques and applications in vision.

In the standard case of MRF with pairwise potentials, we have an undirected graph , where  represents a set of pixels (or superpixels) from an input image, and  denotes a set of edges consisting of unordered pairs of nodes indicating adjacency relations. We consider the problem of minimizing the following energy function:

(1)

Here, we use to denote the label of node , which belongs to a pre-defined finite set representing classes, where . is usually called unary potential, and is derived from the observed data. It measures how well label  fits node . is often referred to as pairwise potential. It measures the cost of assigning labels to adjacent nodes . Typically, it is used to impose spatial smoothness or to align the solution boundaries to image edges. The goal is to find a labeling  (i.e., a mapping from  to ) that minimizes . The Potts function , where is a constant, and is if its argument is true and otherwise, is widely used, among many other functionals.

Minimizing energy (1) is a difficult problem (

-hard in general). In the case of an undirected graph, and by introducing binary variables

, , , which indicate whether node is assigned label ( in this case), the corresponding ILP formulation with Potts function boils down to:

(2)
(2a)
(2b)

where denotes the unary data term for label  and node , and  is a positive parameter weighting the contribution of the smoothness term. Constraint (2a) enforces that each node is assigned exactly one label.

Since (2) is -hard and difficult to solve to optimality, it is common in vision to solve the corresponding LP relaxation [7, 8]. There have been works on solving approximations of (2), for instance, message passing algorithms [9, 10] and -expansion [11] with guaranteed approximation ratios. The corresponding condition for -expansion is nonnegative edge weights and , for all labels . On the other hand, it is also important to solve (2) to optimality (even off-line), thus providing ground-truth benchmarks for those fast approximate algorithms.

The standard model in (2), which combines unary and pairwise potentials, can impose only a limited class of constraints on the solution. Therefore, there is an ongoing research effort in computer vision towards embedding high-order constraints in MRFs. These includes, for instance, region connectivity [1, 2, 3], shape convexity [12], curvature regularization [13] and shape compactness [14], among other high-order priors. In this paper, we investigate exact region connectedness priors. More precisely, we are interested in solving (2) to global optimality, while adding a global (high-order) potential function to (2) to explicitly enforce the connectivity of each label (to be made more precise in Sec. 2). A -label partitioning of the image in this paper is a partition of into connected subgraphs such that , and , . Without loss of generality, we assume that segment (subgraph) is assigned the label . Enforcing the connectivity potential itself is proven to be -hard in [3].

1.1 Related Works

Image segmentation under approximate connectivity constraints has been considered in [3], where a binary MRF is solved. Exact connectivity is not considered in [3]

. Instead, a simplified version of the problem is proposed, where only a given (user-provided) pair of nodes must be connected. Following this assumption, the problem is solved with a heuristic-based graph cut algorithm

[15], obtaining connected foreground (binary segmentation).

Exact global connectivity potentials are formulated as an ILP in [2], where connected subgraph polytopes are introduced. Due to the high computational cost of solving the corresponding -hard problem, the work in [2] examined only LP relaxations of the ensuing ILP. Although the general formulation works for multi-label MRFs, the authors applied it only to binary MRF problems. In [16], the authors optimized exactly a linear (unary-potential) objective subject to connectivity constraint in a binary segmentation problem. It solves two instances of medical benchmark datasets to optimality for the first time. However, the model does not apply to the general multi-label pairwise MRF objective in (1), which is of wide interest in vision applications. Finally, it is worth mentioning that the subgraph connectivity problem also plays an important role in the operations research community, and has been applied, for instance, to the forest planning problem [17], where each subregion of the forest is constrained to be connected.

1.2 Contribution

This paper investigates multi-label MRFs with exact connectivity constraints. To solve the ensuing ILP problem, we propose a branch-and-cut method, which provably finds globally optimal solutions. The method could provide feasible solutions with a guarantee on suboptimality even if we terminate it earlier. Unlike [1, 2], which examines LP relaxations of the initial ILP, our method provides global optimality guarantee. Different from [3], we consider exact connectivity and we do not reduce the problem to connectivity between a given pair of points. The proposed ILP is quite general, and can be applied as a post-processing method on top of any existing multi-label segmentation approach. As it provides globally optimal solution, it can be used off-line to generate ground-truth labeling, which serves as quality check for any fast on-line algorithm. Furthermore, it can be used to generate ground-truth proposals for state-of-the-art weakly supervised semantic segmentation techniques, e.g., those based on partial scribble-based annotations [18].

2 Connected Subgraph Polytopes

In this section, we introduce the convex hull of the set of all connected subgraphs, where a connected subgraph consists of nodes with the same label that are connected. We call a node active if , e.g., if it is labeled as foreground.

Connected Subgraph Polytope. Given a connected, undirected graph , let , where and . Recall that a subgraph is connected if , a path in that connects and . Then, denotes the finite set of connected subgraphs of , and we call the convex hull of the connected subgraph polytope of , denoted by . It was proven in [19] that optimizing a linear function over is -hard.

Vertex-Separator Set. Given an undirected graph , for any pair of active nodes , the set is called a vertex-separator set with respect to if the removal of from disconnects and in . As an additional definition, a set is said to be a minimal vertex-separator set if it is a vertex-separator set with respect to a node pair in  while any strict subset is not.

Let be the collection of all vertex-separator sets in , and be the subsets of minimal vertex-separator sets.

Following [2], we can describe with the class of linear inequalities

(3)

where , . Precisely, if two nodes and are active (left hand side of (3) becomes ), they are not allowed to be separated by any set of inactive nodes of (at least one node in any must be active).

The convex hull of a finite set is the tightest possible convex relaxation, and facet-defining inequalities are true facets of the convex hull. In [2], the authors prove inequalities (3) are facet-defining for if is replaced by . However, the number of such constraints is exponential in .

Rooted case. In this paper, we require the user to input a scribble for each label, so that at least one node is identified within each label. Let denote the root node for the label (we use the first node of the scribble). Then, it suffices to check connectivity of every active node to the root node instead of all pairs of active nodes. Thus, constraints (3) become

(4)
Theorem 1.

It is still -hard to optimize over the connected subgraph polytope even if one root node is given.

Proof.

The proof can be found in the supplementary materials. ∎

In practice, the number of constraints (4) is still exponential in (as the number of vertex-separator set is), hence they cannot be considered all simultaneously for graphs of large sizes. However, given a labeling , we can identify a subset of violated connectivity constraints of type (4) in polynomial time and iteratively add them to the ILP while searching for new integer solutions. This is known as the cut generation approach. We will look into this in detail in Sec. 4.1.

3 MRFs with Connectivity Constraints

3.1 Proposed model: ILP-PC

Let denotes the observed image feature (e.g., color) at spatial location . We assume the user inputs scribbles as seeds for the labels, as shown in the left image of Fig. 4. Assuming image observations follow a piecewise constant model within each region111We assume a piecewise constant model for simplicity. However, our formulation extends to any other probabilistic assumptions of observation models., let denotes the image average of seeds within label . In this case, unary potential evaluates how well label fits node .

Let denotes the connected subgraph polytope of label , and . By introducing two nonnegative variables and to model , the ILP of our multi-label MRF with connectivity constraints becomes:

(5)
(5a)
(5b)
(5c)
(5d)
(5e)
(5f)

where constraints (5d) can be expressed as the rooted vertex-separator constraints (4). The transformation holds because (5) is a minimization problem, and will only induce and .

In the case of a superpixel graph, where a superpixel contains similar pixels in terms of color or texture, we represent relations between neighboring superpixels by defining the corresponding Region Adjacency Graph (RAG) , where contains edges between pairs of adjacent superpixels. We multiply the unary data term by and the pairwise term by . Here, denotes the number of pixels contained in node (superpixel) , and represents the number of neighboring pixels between node and .

3.2 ILP-PCB: ILP-PC with background label

If a clear background (not necessarily connected) exists in the given image, the connectivity constraints can be ignored on the specific label, which we call the background label. This is a reasonable assumption in many cases, such as the black-region background in the left image of Fig. 5. In this example, the background has disconnected components. Fig. 5 depicts results with and without background label.

Figure 1: -Nearest cut generation strategy. Active nodes are shown in black, and the two separator sets are marked in red and blue.

4 Solution Techniques

4.1 Towards global optima: the branch-and-cut method

The most widely used exact method for solving an ILP is branch and cut, namely a sophisticated combination of the branch and bound and the cutting plane algorithms [20]. In this section, we focus on the cutting plane method because the branch-and-bound method is implemented by default in any modern ILP solver. We recall that the ILP gap of a minimization ILP problem is computed as , where denotes the best integer solution and the LP-relaxation value. The ILP solver terminates if the ILP gap is very small, or if the time limit is reached.

We are interested in exact connectivity and we focus on the rooted case (4). We concentrate on enforcing the connectivity constraints for one label only (e.g., ). Then, the same approach will be repeated for other labels until they are all connected (in the case of a background label, we ignore its connectivity).

The basic idea is to omit (5d) initially, explore the branch-and-bound tree of system (5) until an integer solution is found and then check the feasibility of this solution (i.e., connectivity). If infeasible, violated constraints (5d) are identified (by solving the so-called the separation problem) and added to (5) to cut off the infeasible solution. This procedure is iterated until  is connected.

We treat individual connected components (see Fig. 1) as one entity, since establishing connectivity between all nodes in this component and automatically connects all the nodes. Then, the separation problem boils down to finding a vertex-separator set between each disconnected active component and the root component (containing ) in the current solution.

At the heart of the above technique is that only a subset of connectivity constraints (5d) will be active at the optimum of (5), i.e., polynomially many should be enough for the algorithm to converge to the optimal solution. However, depending on the choice of the inequalities at each step, we may require a different number of such cutting planes and the number of iterations varies.

The separation problem and cutting plane selection. Among the many ways of separating and selecting the violated constraints (5d), we choose the so-called -Nearest strategy. Note that, although the separation problem for constraints (5d) is generally -hard (see Sec. 2), it is polynomial to separate integer infeasible solutions, for example, by means of the following algorithm. Precisely, we run a breath-first search for any active component to collect the (disjoint) vertex separator sets  () composed of all nodes with identical distance. The search terminates if equals the number of nodes in or if another active node is reached. The idea is illustrated in Fig. 1, where active nodes are shown in black and denotes the root node. The two separator sets are marked in red and blue. Here , because it reaches the number of nodes in .

The -Nearest strategy is reported in [16] to be one of the most successful (among five) in terms of solved instances and computational efficiency. We will adopt this vertex separation strategy in Sec. 5.

4.2 -: a region fusion based heuristic

To improve the efficiency of the ILP solver while solving problem (5) with the method described in the previous section, we calculate an initial feasible solution with a heuristic, called -, which will be used as an upper bound of the branch-and-bound tree. On the one hand, this helps to prune a lot of unnecessary branching nodes. On the other hand, the solver can provide an optimality gap to the initial solution, by solving the LP relaxation of the ILP, which will serve as a lower bound to the problem.

We adopted the idea for the heuristic from [21], which is basically a local greedy algorithm to solve the discrete Potts model [22]. It works by iteratively merging groups of nodes.

In the beginning, each scribble of nodes (superpixels) and every node not covered by any scribbles are in their own groups. Then for every iteration, we merge two neighboring groups, if the following condition holds and the merging does not result in the nodes of two different scribbles being in the same group:

(6)

where denotes the number of pixels in group (segments) , denotes the number of neighboring pixels (boundary length) of two groups and , and the mean of image data (e.g., color) within group . By increasing parameter  in (6) in every iteration, we terminate the algorithm when exactly groups remain. It is shown in [21] that the following exponentially growing strategy of gives the best results.

where iter is the current iteration number, and is the regularization parameter for the Potts model. We will show in Sec. 5 that - is fast and generates good results most of the time, sometimes even optimal.

5 Experiments

In this paper, all computational experiments were performed using Cplex , on a Intel i5-4570 quad-core machine, with

GB RAM. We show experiments on medical images, where the unary potentials are based on the probability maps of given labels, which were trained using convolutional neural networks (CNN)

[23]. The sizes range from to .

We further use the Berkeley Segmentation Dataset [24] (BSDS500, image size ) and the PASCAL VOC 2012 set [25] (PASCAL, image size around ). We first apply the SLIC [26] superpixel algorithm to get an over-segmentation, with the number of superpixel around .

Using superpixels has several advantages. First, the complexity of the optimization problem is drastically reduced with only a negligible segmentation error. Second, the information in each superpixel is more discriminative, and also overcomes the case of outliers. As shown in a recent superpixel algorithms survey paper 

[27], a few advanced superpixel algorithms can achieve very accurate over-segmentation results with around suerpixels.

We conduct a comprehensive comparison of the following different optimization models:

  • ILP-PC. Our proposed ILP formulation (5) of multi-label MRF, under the global connectivity constraints.

  • LP-PC: The LP relaxation of ILP-PC, which was introduced in  [2, 16].

  • -: Our proposed region fusion based heuristic, which was motivated by  [21] and modified to generate exactly connected segments.

  • ILP-P: The ILP formulation of (5) without connectivity constraints (5d), which is widely used in vision (e.g., graph cuts).

  • ILP-PCB: ILP-PC with the “background” label marked by the user, where this special label is not required to be connected.

In this section, if there is no further explanation, the default setting for the pairwise potential is ,  secs for the time limit, and for the - parameter . We adopt - to provide initial solution for the ILP solver. When we report energy , we use the objective function in (5).

Figure 2: Ground-truth generation on images taken from PASCAL. for the third image, , and secs, , and .

5.1 Ground-truth generation

Our proposed ILP solver is -hard, but provides global optimal solution for the multi-label MRFs with connectivity prior. Thus, it could be used off-line to generate ground-truth labeling, which serves as quality check for any fast on-line algorithms.

We conduct experiments on two instances taken from PASCAL, where the scribbles of all training images are online available and provided by ScribbleSup [18]. We set equals for the first two image and for the third. ILP-PC takes only and secs on the first two instances and secs on the third to get the optimal solution. The optimal energy are , and respectively.

5.2 Detailed comparison of different models

5.2.1 Medical images with probability maps

(a) Medical image
(b) Probability map
(c) User scribbles
(d) -, 0.83s.
(e) ILP-PC, 100s.
(f) ILP-PCB, 100s.
(g) LP-PC, 1.19s.
(h) ILP-P, 0.46s.
(i) Ground truth.
Figure 3: Comparison of 5 models on a medical image, where user scribbles are depicted on the probability map. Energy is reported in Sec. 5.2.1, and numbers denote the time spent. In LP-PC, of the pixels remains unlabeled , colored in white. Both ILP-PC and ILP-PCB have ILP gap.

— X[c] — X[c] — X[c] — X[c] — X[c] — - & ILP-PC & ILP-PCB & LP-PC & ILP-P
864.5 & 854 & 854 & 829.5 & 826.8

Table 1: Energies of proposed models on an MRI image.

We report a medical image segmentation example, where unary potentials are based on the probability maps of given labels, which were trained using convolutional neural networks (CNN) [23]. The purpose here is to obtain a binary (two-region) segmentation of a magnetic resonance image (MRI), which depicts the abdominal aorta [14]. In this example, the CNN probability maps yielded unsatisfying disconnected region due to imaging noise, the lack of boundary contrast and limited training information.

The input image is of size , and the computation time is reported in Fig. 3, where ILP-PC and ILP-PCB both failed to converge. - result is of high quality, within of the best solution found by ILP-PC in seconds. The energy of all models is reported in table 1.

As we see in Fig. 3, LP-PC has fractional solution (depicted in white). Although a post-processing rounding heuristic can be applied, it is not guaranteed even to find a feasible solution. ILP-P gives two separated regions, which is far away from the ground truth. We notice that LP-PC and ILP-P give lower energy than ILP-PC. This is because both of them are relaxations for ILP-PC and, therefore, provide lower bounds. The inclusion of background label is not beneficial in this example.

5.2.2 Superpixels of BSDS500

In this section, we introduce another model ILP-PCW, which is ILP-PC without the initial solution of -. The purpose is to test whether the ILP solver is able to achieve good results by itself.

(a) Input with user scribbles.
(b) -, 0.04s, E=16088.
(c) ILP-PC, 100s, E=15804.9.
(d) LP-PC, 0.27s, E=15560.9.
(e) ILP-P, 0.08s, E=15232.5.
(f) ILP-PCW, 61s, E=15804.9.
Figure 4: Comparison of models on BSDS, including ILP-PCW ( ILP-PC without initial solution from -). Note that of the nodes remains unlabeled in LP-PC, colored in white. The - solution is within of best solution.

Fig. 4 depicts an example, where ILP-PC with - does not converge within the time limit while ILP-PCW finds the provably global optimal solution. The solution time and the energy are reported in the figure. The energy of the starting solution provided by - is very good, within of the optimal solution found by ILP-PCW in seconds. Note that ILP-PC and ILP-PCW give the same energy, meaning they found the same solution, but ILP-PC failed to get the tightest lower bound, having an ILP gap of . A closer look into the log file of Cplex shows that, given the good initial solution of -, ILP-PC found this best solution in less than sec, while ILP-PCW takes secs.

The inclusion of the integrality constraints (5c) and the connectivity priors greatly improve solution quality. As many as superpixel values of LP-PC are factional. In ILP-P, the green and black labels have several disconnected regions, resulting in a worse solution.

5.2.3 More experiments on BSDS500 images

More experiments on BSDS500 images are shown in Fig. 6. In the first column, the pairwise term is set to to encourage thin branches of the tree, while all other parameters remain at their default values (). We draw much fewer brushes in the right two columns, to show the robustness of our model (to be discussed in Sec. 5.4 with more details). We observe that - gives good results in the right two images, while not being satisfying in the left two cases. Our proposed model ILP-PC achieves the best overall results.

5.3 Quantitative comparison of different models

In this section, we give a detailed analysis of the different models with respect to energy, computational time and parameters. They are based on computational experiments of images from BSDS500 and medical images from [14].

5.3.1 Statistics of models.

We report the average running time of all models in the second row of Table 2, where the time limit is sec. The average ILP optimality gap is shown in the third row, where “Null” means no ILP gap exists (because they are not an ILP). We can see that the ILP-PC on average takes seconds, and the ILP gap is . Moreover, the inclusion of the “background label” (ILP-PCB) helps in term of both speed and ILP gap. There exist two instances where ILP-PCB reduces the time of (time limit reached) secs from ILP-PC to less than one sec. This gain results from “relaxing” one label to be non-connected. It is also surprising to see that ILP-P with only pairwise prior is also very efficient.

— X[c]— X[c] — X[c] — X[c] — X[c] — X[c] — &- & ILP-PC &ILP-PCB & LP-PC &ILP-P
Time &0.7 & 62.3 & 39.2 & 1.4 & 0.3
Gap &Null & 3.7% & 1.9% & Null & 0

Table 2: Time and optimality gap of 5 proposed models.

Apart from the above statistics, we also report that among all tested images an average of pixels found by LP-PC remain unlabeled (fractional solutions). Hence, we argue that LP-PC is not applicable in practice.

5.3.2 ILP-PC against -

In the conducted experiments, - is found to be fast and it provides an initial solution to the ILP solver. ILP-PC adopts this initial solution, and seeks for better ones as the branch-and-cut tree proceeds. On average, the ILP-PC is able to improve quality of the initial solution (provided by -) within the time limit. Moreover, it could provide any feasible solution a lower bound (thus a sub-optimality guarantee) upon solving the LP-relaxation of the ILP.

5.3.3 ILP-PC with secs time limit

We further conduct experiments on the same instances with a time limit of ILP-PC set to secs. - is again applied as pre-processing for the solver. We observe that ILP could improved out of the instances, and increase on average quality of initial solution in just secs. Hence, we argue that our proposed ILP model can be beneficial even within very short time, and thus applicable in much wider scenarios.

Figure 5: Different user scribbles on the same image. Second row: ILP-PC with E=9544.2 and 13422.8, both reaching 100 seconds time limit. Bottom row: ILP-PCB with E=8615.3 and 10482.3, t= 2.1s and 0.4s. Note that the background label (shown in black) can be disconnected, while the other labels (blue and green) are connected.

5.4 Analysis of different user scribbles

The user scribbles are used to learn the average color of each label, which is used in the ILP as the unary term. They also enforce hard constraints (5f) into the ILP (5), which help fixing some of the binary variables, thus pruning the branch-and-bound search trees within the ILP solver. Moreover, in case of difficult situations, scribbles can also be used to exclude outliers from one label, such as in Fig.3. We show in Fig. 5 that changing the scribbles does not alter significantly the results. While ILP-PC reaches the time limit in both cases, ILP-PCB gets the reported optimal solution in only and secs. The energy differences between the two cases are due to two factors: different scribbles resulted not only into different hard constraints, but also different unary potentials.

Figure 6: More experiments on BSDS500 images. The pairwise term is set to to encourage thin branches of the tree in the first column. The user scribbles in the right two columns are fewer, to show the robustness of our model. The white pixels in LP-PC denote fractional solutions, and ILP-P is without connectivity constraints, thus allowing disconnected regions with the same label.

6 Conclusion

Recent years’ algorithmic advances in Integer Programming plus the hardware improvements have resulted in a enormous speedup in solving ILPs. We revisit the ILP of the multi-label MRF with connectedness priors, and propose an exact branch-and-cut approach that enforce the connectivity constraints on the fly through cutting plane generation. A fast region fusion based heuristic is designed to provide a good initial solution. The solver provides a nearly-optimal solution with a guarantee on the sub-optimality even if we terminate it earlier.

The ILP model can be applied to generate ground-truth labeling and segmentation offline, thus providing a quality assessment for any fast algorithm.

It can also be applied as a post-processing method on top of any existing multi-label segmentation approach. Hence, the advantage of ILP is two-fold. On the one hand, it provides a guarantee (lower bound) for any given initial solution. On the other hand, it seeks for better solutions during its search in the branch-and-bound tree, and it is beneficial even within very short time.

In this paper, we demonstrated the power and usefulness of our model by some experiments on the PASCAL, BSDS500 dataset, and medical images with trained probability maps. We have shown that with moderate-sized images or superpixels of large ones, our model achieves the best overall performance, yielding a provably global optimum in some instances.

References

  • [1] Nowozin, S., Lampert, C.: Global interactions in random field models: A potential function ensuring connectedness. SIAM Journal on Imaging Sciences 3(4) (Dec 2010) 1048–1074
  • [2] Nowozin, S., Lampert, C.H.: Global connectivity potentials for random field models.

    In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. (Jun 2009) 818–825

  • [3] Vicente, S., Kolmogorov, V., Rother, C.: Graph cut based image segmentation with connectivity priors. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition. (Jun 2008) 1–8
  • [4] Kappes, J.H., Andres, B., Hamprecht, F.A., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B.X., Kröger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. International Journal of Computer Vision 115(2) (Nov 2015) 155–184
  • [5] Kindermann, R., Snell, J.L.: Markov random fields and their applications. Providence, R.I. : American Mathematical Society (1980) References p.133-142.
  • [6] Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(6) (Jun 2008) 1286–80
  • [7] Komodakis, N., Paragios, N., Tziritas, G.: MRF energy minimization and beyond via dual decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3) (Mar 2011) 531–552
  • [8] Komodakis, N., Tziritas, G.: Approximate labeling via graph cuts based on linear programming. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(8) (Aug 2007) 1436–1453
  • [9] Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(10) (Oct 2006) 1568–1583
  • [10] Yedidia, J., Freeman, W., Weiss, Y.: Understanding belief propagation and its generalizations.

    In: Exploring artificial intelligence in the new millennium.

    Morgan Kaufmann Publishers Inc. (2003) 239–269
  • [11] Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11) (Nov 2001) 1222–1239
  • [12] Gorelick, L., Veksler, O., Boykov, Y., Nieuwenhuis, C.: Convexity shape prior for binary segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(2) (Feb 2017) 258–271
  • [13] Nieuwenhuis, C., Toeppe, E., Gorelick, L., Veksler, O., Boykov, Y.: Efficient squared curvature. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. (June 2014) 4098–4105
  • [14] Dolz, J., Ayed, I.B., Desrosiers, C.: Unbiased shape compactness for segmentation. In: Medical Image Computing and Computer Assisted Intervention - MICCAI (1). (Sep 2017) 755–763
  • [15] Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9) (Sep 2004) 1124–1137
  • [16] Rempfler, M., Andres, B., Menze, B.: The minimum cost connected subgraph problem in medical image analysis. In: Medical Image Computing and Computer Assisted Intervention - MICCAI (3). (Oct 2016) 397–405
  • [17] Carvajal, R., Constantino, M., Goycoolea, M., Vielma, J.P., Weintraub, A.: Imposing connectivity constraints in forest planning models. Operations Research (Jul 2013) 824
  • [18] Lin, D., Dai, J., Jia, J., He, K., Sun, J.: Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 3159–3167
  • [19] Karp, R.M.: Maximum-weight connected subgraph problem. http://www.cytoscape.org/ISMB2002/ (2002)
  • [20] Lodi, A. In: Mixed Integer Programming Computation. Springer Berlin Heidelberg, Berlin, Heidelberg (2010) 619–645
  • [21] Nguyen, R.M.H., Brown, M.S.: Fast and effective l0 gradient minimization by region fusion. In: Proceedings of the IEEE International Conference on Computer Vision. (Dec 2015) 208–216
  • [22] Potts, R., Domb, C.: Some generalized order-disorder transformations. Proceedings of the Cambridge Philosophical Society 48 (1952) 106–109
  • [23] Dolz, J., Desrosiers, C., Ayed, I.B.: 3d fully convolutional networks for subcortical segmentation in mri: A large-scale study. NeuroImage (Apr 2017)
  • [24] Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5) (May 2011) 898–916
  • [25] Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88 (2010) 303–338
  • [26] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11) (Nov 2012) 2274–2282
  • [27] Stutz, D., Hermans, A., Leibe, B.: Superpixels: An evaluation of the state-of-the-art. Computer Vision and Image Understanding 166(Supplement C) (Jan 2018)

7 Appendix

7.1 Proof for Theorem

Proof.

The problem of enforcing connectivity on one label in [3] is proved to be -hard. Suppose by fixing a root node, this problem becomes polynomial solvable. Then, we can randomly assign a node to be the root node, and solve the resulting problem in polynomial time. Since there are () possible root nodes, by trying out all possible root nodes (still in polynomial time), we are sure to find the optimal solution out of optimization problems. Thus a contradiction. ∎