Similar Elements and Metric Labeling on Complete Graphs

1 Similar Elements

Let $X$ be a (possibly infinite) set and $d$ be a metric on $X$ . Let $S_{1}, \dots, S_{n}$ be $n$ finite subsets of $X$ . The goal of the similar elements problem is to select an element from each set $S_{i}$ such that the selected elements are close to each other under the metric $d$ . One motivation is for discovering something in common among the sets $S_{1}, \dots, S_{n}$ even when they have empty intersection.

We formalize the problem as the minimization of the sum of pairwise distances among selected elements. Let $x = (x_{1}, \dots, x_{n})$ with $x_{i} \in S_{i}$ . Define the similar elements objective as,

c (x) = \sum 1 \leq i, j \leq n d (x_{i}, x_{j}) .

(1)

Let $x^{*} = {argmin}_{x} c (x)$ be an optimal solution for the similar elements problem.

Optimizing $c (x)$ appears to be difficult, but we can define easier problems if we ignore some of the pairwise distances in the objective. In particular we define $n$ different “star-graph” objective functions as follows. For each $1 \leq r \leq n$ define the objective $c^{r} (x)$ to account only for the terms in $c (x)$ involving $x_{r}$ ,

c^{r} (x) = \sum j \neq r d (x_{r}, x_{j}) .

(2)

Let $x^{r} = {argmin}_{x} c^{r} (x)$ be an optimal solution for the optimization problem defined by $c^{r} (x)$ . We can compute $x^{r}$ efficiently using a simple form of dynamic programming, by first computing $x_{r}^{r}$ and then computing $x_{j}^{r}$ for $j \neq r$ .

x_{r}^{r} = argmin x_{r} \in S_{r} \sum j \neq r min x_{j} \in S_{j} d (x_{r}, x_{j}),

(3)

x_{j}^{r} = argmin x_{j} \in S_{j} d (x_{r}^{r}, x_{j}) .

(4)

Each of the $n$ “star-graph” objective functions leads to a possible solution. We then select from among the solutions $x^{1}, \dots, x^{n}$ as follows,

	$^r$	$=$	$argmin 1 \leq r \leq n c^{r} (x^{r}),$		(5)
	$^x$	$=$	$x^{r} .$		(6)

Theorem 1.

The algorithm described above finds a 2-approximate solution for the similar elements problem. That is,

c (^x) \leq 2 c (x^{*}) .

Proof.

First note that,

c (x) = n \sum r = 1 c^{r} (x) .

Since the minimum of a set of values is at most the average, and $x^{r}$ minimizes $c^{r} (x)$ ,

min 1 \leq r \leq n c^{r} (x^{r}) \leq \frac{1}{n} n \sum r = 1 c^{r} (x^{r}) \leq \frac{1}{n} n \sum r = 1 c^{r} (x^{*}) = \frac{1}{n} c (x^{*}) .

By the triangle inequality we have

c (x) = \sum 1 \leq i, j \leq n d (x_{i}, x_{j}) \leq \sum 1 \leq i, j \leq n (d (x_{i}, x_{r}) + d (x_{r}, x_{j})) = 2 n n \sum l = 1 d (x_{r}, x_{l}) = 2 n c^{r} (x) .

Therefore

c (^x) \leq 2 n c^{^r} (^x) = 2 n min 1 \leq r \leq n c^{r} (x^{r}) \leq 2 c (x^{*}) .

∎

To analyze the running time of the algorithm we assume the distances $d (p, q)$ between pairs of elements in $S = S_{1} \cup \dots \cup S_{n}$ are either pre-computed and given as part of the input, or they can each be computed in $O (1)$ time.

Let $k = {max}_{1 \leq i \leq n} | S_{i} |$ . The first stage of the algorithm involves $n$ optimization problems that can be solved in $O (n k^{2})$ time each. The second stage of the algorithm involves selecting one of the $n$ solutions, and takes $O (n^{2})$ time.

Remark 2.

If each of the sets $S_{1}, \dots, S_{n}$ has size at most $k$ the running time of the approximation algorithm for the similar elements problem is $O (n^{2} k^{2})$ .

The bottleneck of the algorithm is the evaluation of the minimizations over $x_{j} \in S_{j}$ in (3) and (4). This computation is equivalent to a nearest-neighbor computation, where we want to find a point from a set $S \subseteq X$ that is closest to a query point $q \in X$ . When the nearest-neighbor computation can be done efficiently (with an appropriate data structure) the running time of the similar elements approximation algorithm can be reduced.

2 Metric Labeling on Complete Graphs

Let $G = (V, E)$ be an undirected simple graph on $n$ nodes $V = {1, \dots, n}$ . Let $L$ be a finite set of labels with $| L | = k$ and $d$ be a metric on $L$ . For $i \in V$ let $m_{i}$ be a non-negative function mapping labels to real values. The unweighted metric labeling problem on $G$ is to find a labeling $x = (x_{1}, \dots, x_{n}) \in L^{n}$ minimizing

c (x) = \sum i \in V m_{i} (x_{i}) + \sum {i, j} \in E d (x_{i}, x_{j}) .

(7)

Let $x^{*} = {argmin}_{x} c (x)$ . This optimization problem can be solved in polynomial time using dynamic programming if $G$ is a tree. Here we consider the case when $G$ is the complete graph and give an efficient 2-approximation algorithm based on the solution of several metric labeling problems on star graphs.

For each $r \in V$ define a different objective function, $c^{r} (x)$ , corresponding to a metric labeling problem on a star graph with vertex set $V$ rooted at $r$ ,

c^{r} (x) = \sum i \in V \frac{m_{i} (x_{i})}{n} + \sum j \in V ∖ {r} \frac{d (x_{r}, x_{j})}{2} .

(8)

Let $x^{r} = {argmin}_{x} c^{r} (x)$ . We can solve this optimization problem in $O (n k^{2})$ time using a simple form of dynamic programming. First compute an optimal label for the root vertex using one step of dynamic programming,

x_{r}^{r} = argmin x_{r} \in L ⎛ ⎝ \frac{m_{r} (x_{r})}{n} + \sum j \in V ∖ {r} min x_{j} \in L (\frac{m_{j} (x_{j})}{n} + \frac{d (x_{r}, x_{j})}{2}) ⎞ ⎠ .

(9)

Then compute $x_{j}^{r}$ for $j \in V ∖ {r}$ ,

x_{j}^{r} = argmin x_{j} \in L (\frac{m_{j} (x_{j})}{n} + \frac{d (x_{r}^{r}, x_{j})}{2}) .

(10)

Optimizing each $c^{r} (x)$ separately leads to $n$ possible solutions $x^{1}, \dots, x^{n}$ , and we select one of them as follows,

	$^r$	$=$	$argmin r \in V c^{r} (x^{r}),$		(11)
	$^x$	$=$	$x^{r} .$		(12)

Theorem 3.

The algorithm described above finds a 2-approximate solution for the metric labeling problem on a complete graph. That is,

c (^x) \leq 2 c (x^{*}) .

Proof.

First note that,

c (x) = n \sum r = 1 c^{r} (x) .

Since the minimum of a set of values is at most the average, and $x^{r}$ minimizes $c^{r} (x)$ ,

min 1 \leq r \leq n c^{r} (x^{r}) \leq \frac{1}{n} n \sum r = 1 c^{r} (x^{r}) \leq \frac{1}{n} n \sum r = 1 c^{r} (x^{*}) = \frac{1}{n} c (x^{*}) .

Since $d$ is a metric and $m_{i}$ is non-negative,

$c (x)$	$=$	$\sum i \in V m_{i} (x_{i}) + \sum {i, j} \in E d (x_{i}, x_{j})$
	$=$	$\sum i \in V m_{i} (x_{i}) + \sum (i, j) \in V^{2} \frac{d (x_{i}, x_{j})}{2}$
	$\leq$	$\sum i \in V m_{i} (x_{i}) + \sum (i, j) \in V^{2} (\frac{d (x_{i}, x_{r})}{2} + \frac{d (x_{r}, x_{j})}{2})$
	$=$	$\sum i \in V m_{i} (x_{i}) + 2 n \sum l \in V ∖ {r} \frac{d (x_{r}, x_{l})}{2}$
	$\leq$	$2 n \sum i \in V \frac{m_{i} (x_{i})}{n} + 2 n \sum l \in V ∖ {r} \frac{d (x_{r}, x_{l})}{2}$
	$=$	$2 n c^{r} (x) .$

Therefore

c (^x) \leq 2 n c^{^r} (^x) = 2 n min 1 \leq r \leq n c^{r} (x^{r}) \leq 2 c (x^{*}) .

∎

The first stage of the algorithm involves $n$ optimization problems that can be solved in $O (n k^{2})$ time each. The second stage involves selecting one of the $n$ solutions, and takes $O (n^{2})$ time.

Remark 4.

The running time of the approximation algorithm for the metric labeling problem on complete graphs is $O (n^{2} k^{2})$ .

Acknowledgments

We thank Caroline Klivans, Sarah Sachs, Anna Grim, Robert Kleinberg and Yang Yuan for helpful discussions about the contents of this report. This material is based upon work supported by the National Science Foundation under Grant No. 1447413.

References

[1] Dan Gusfield. Efficient methods for multiple sequence alignment with guaranteed error bounds. Bulletin of Mathematical Biology, 55(1):141–154, 1993.
[2] Jon Kleinberg and Eva Tardos. Approximation algorithms for classification problems with pairwise relationships: Metric labeling and markov random fields. Journal of the ACM, 49(5):616–639, 2002.
[3] Oded Maron and Aparna Lakshmi Ratan.
Multiple-instance learning for natural scene classification.
In International Conference on Machine Learning, volume 98, pages 341–349, 1998.
[4] Sarah Sachs. Similar-part approximation using invariant feature descriptors. Undergraduate Honors Thesis, Brown University, 2016.

Similar Elements and Metric Labeling on Complete Graphs

Authors

A (Slightly) Improved Approximation Algorithm for Metric TSP

A historical note on the 3/2-approximation algorithm for the metric traveling salesman problem

A constant-ratio approximation algorithm for a class of hub-and-spoke network design problems and metric labeling problems: star metric case

An Extension of Plücker Relations with Applications to Subdeterminant Maximization

Separating Variables in Bivariate Polynomial Ideals

On ultrametric 1-median selection

Image Labeling by Assignment

1 Similar Elements

Theorem 1.

Proof.

Remark 2.

2 Metric Labeling on Complete Graphs

Theorem 3.

Proof.

Remark 4.

Acknowledgments

References

Similar Elements and Metric Labeling on Complete Graphs

Authors

Related Research

A (Slightly) Improved Approximation Algorithm for Metric TSP

A historical note on the 3/2-approximation algorithm for the metric traveling salesman problem

A constant-ratio approximation algorithm for a class of hub-and-spoke network design problems and metric labeling problems: star metric case

An Extension of Plücker Relations with Applications to Subdeterminant Maximization

Separating Variables in Bivariate Polynomial Ideals

On ultrametric 1-median selection

Image Labeling by Assignment

This week in AI

1 Similar Elements

Theorem 1.

Proof.

Remark 2.

2 Metric Labeling on Complete Graphs

Theorem 3.

Proof.

Remark 4.

Acknowledgments

References