Distributed stochastic optimization with gradient tracking over strongly-connected networks

I Introduction

In the era of data deluge, where it is particularly difficult to store and process all data on a single device/node/processor, distributed schemes are becoming attractive for inference, learning, and optimization. Distributed optimization over multi-agent systems, thus, has been of significant interest in many areas including but not limited to machine learning

[1, 2], big-data analytics [3, 4], and distributed control [5, 6]. However, the underlying algorithms must be designed to address practical limitations and realistic scenarios. For instance, with the computation and data collection/storage being pushed to the edge devices, e.g., in Internet of Things (IoT), the data available for distributed optimization is often inexact. Moreover, the ad hoc nature of setups outside of data centers requires the algorithms to be amenable to communication protocols that are not necessarily bidirectional. The focus of this paper is to study and characterize distributed optimization schemes where the inter-agent communication is restricted to directed graphs and the information/data is inexact.

In particular, we study distributed stochastic optimization over directed graphs and propose the $S$ - $A B$ algorithm to minimize a sum of local cost functions. The $S$ - $A B$ algorithm assumes access to a stochastic first-order oracle ( $S F O$ ), i.e., when an agent queries the $S F O$

, it gets an unbiased estimate of the gradient of its local cost function. In the proposed approach, each agent makes a weighted average of its own and its neighbors’ solution estimates, and simultaneously incorporates its local gradient estimate of the global cost function. The exchange of solution estimates is performed over a row-stochastic weight matrix. In parallel, each agent maintains its own estimate of the gradient of the global cost function, by simultaneously incorporating a weighted average of its and its neighbors’ gradient estimates and its local gradient tracking estimate. The exchange of gradient estimates of the global cost function is performed over a column-stochastic weight matrix. Since doubly-stoachstic weights are nowhere used,

S

A B

is an attractive solution that is applicable to arbitrary, strongly-connected graphs.

The main contributions of this paper are as follows: (i) We show that, by choosing a sufficiently small constant step-size, $α$ , $S$ - $A B$ converges linearly to a neighborhood of the global minimizer. This convergence guarantee is achieved for continuously-differentiable, strongly-convex, local cost functions, where each agent is assumed to have access to a $S F O$

and the gradient noise has zero-mean and bounded variance.

(ii) We provide explicit expressions of the appropriate norms under which the row- and column-stochastic weight matrices contract. With the help of these norms, we develop sharp and explicit convergence arguments.

We now briefly review the literature concerning distributed and stochastic optimization. Early work on deterministic finite-sum problems includes [7, 8, 9], while work on stochastic problems can be found in [10, 11]. Recently, gradient tracking has been proposed where the local gradient at each agent is replaced by the estimate of the global gradient [12, 13, 14, 15]. Methods for directed graphs that are based on gradient tracking [16, 17, 18, 19, 14, 15, 20, 21]

rely on separate iterations for eigenvector estimation that may impede the convergence. This issue was recently resolved in

[22, 23], see also [24, 25, 26, 27, 28] for the follow-up work, where eigenvector estimation was removed with the help of a unique approach that uses both row- and column-stochastic weights. Ref. [22] derives linear convergence of the finite-sum problem when the underlying functions are smooth and strongly-convex, however, since arbitrary norms are used in the analysis, the convergence bounds are not sharp. Recent related work on time-varying networks and other approaches can be found in [29, 30, 31, 32, 33], albeit, without gradient tracking. Of significant relevance is [34], where a similar setup with gradient tracking is considered over undirected graphs. We note that

S

A B

generalizes [34] and the analysis in [34] relies on the weight matrix contraction in

2

-norm that is not applicable here.

We now describe the rest of the paper. Section II describes the problem, assumptions, and some auxiliary results. We present the convergence analysis in Section III and the main result in Section IV. Finally, Section V provides the numerical experiments and Section VI concludes the paper.

Basic Notation:

We use lowercase bold letters for vectors and uppercase italic letters for matrices. We use

I_{n}

for the

n \times n

identity matrix, and

1_{n}

for the column of

n

ones. For an arbitrary vector,

x

, we denote its

i

th element by

[x]_{i}

and its smallest element by

x - -

and its largest element by

¯ ¯ ¯ x

. Inequalities involving matrices and vectors are to be interpreted componentwise. For a matrix,

X

, we denote

ρ (X)

as its spectral radius and

X_{\infty}

as its infinite power (if it exists), i.e.,

X_{\infty} = {lim}_{k \to \infty} X^{k}

. For a primitive, row-stochastic matrix,

A

, we denote its left and right eigenvectors corresponding to the eigenvalue of

1

π_{r}

and

1_{n}

, respectively, such that

π_{r}^{⊤} 1_{n} = 1

and

A_{\infty} = 1_{n} π_{r}^{⊤}

. Similarly, for a primitive, column-stochastic matrix,

B

, we have

B_{\infty} = π_{c} 1_{n}^{⊤}

Ii Problem formulation and Auxiliary Results

Consider $n$ agents connected over a directed graph, $G = (V, E)$ , where $V = {1, \dots, n}$ is the set of agents, and $E$

is the collection of ordered pairs,

(i, j), i, j \in V

, such that agent

j

can send information to agent

i

. We assume that

(i, i) \in E, \forall i

. The agents solve the following problem:

P1 : min x \in R^{p} F (x) ≜ \frac{1}{n} n \sum i = 1 f_{i} (x),

(1)

where each $f_{i} : R^{p} \to R$ is known only to agent $i$ . We now formalize the assumptions.

Assumption 1

Each local objective, $f_{i}$ , is $μ$ -strongly-convex, i.e., $\forall i \in V$ and $\forall x, y \in R^{p}$ . Thus, we have

f_{i} (y) \geq f_{i} (x) + \nabla f_{i} (x)^{⊤} (y - x) + \frac{μ}{2} ∥ x - y ∥_{2}^{2} .

Under Assumption 1, the optimal solution for Problem P1 exists and is unique, which we denote as $x^{*}$ .

Assumption 2

Each local objective, $f_{i}$ , is $l$ -smooth, i.e., its gradient is Lipschitz-continuous: $\forall i \in V$ and $\forall x, y \in R^{p}$ , we have, for some $l > 0$ ,

∥ \nabla f_{i} (x) - \nabla f_{i} (y) ∥_{2} \leq l ∥ x - y ∥_{2} .

We make the following assumption on the agent communication graph, which guarantees the existence of a directed path from each agent $i$ to each agent $j$ .

Assumption 3

The graph, $G$ , is strongly-connected.

We consider distributed iterative algorithms to solve Problem P1, where each agent is able to call a stochastic first-order oracle ( $S F O$ ). At iteration $k$ and agent $i$ , given $x_{k}^{i} \in R^{p}$ as the input, $S F O$ returns a stochastic gradient in the form of $g_{i} (x_{k}^{i}, ξ_{k}^{i}) \in R^{p}$ , where $ξ_{k}^{i} \in R^{m}$ are random vectors, $\forall k \geq 0, \forall i \in V$ . The stochastic gradients, $g_{i} (x_{k}^{i}, ξ_{k}^{i})$ , satisfy the following standard assumptions:

Assumption 4

The set of random vectors ${ξ_{k}^{i}}_{k \geq 0, i \in V}$ are independent of each other, and

$E_{ξ_{k}^{i}} [g_{i} (x_{k}^{i}, ξ_{k}^{i}) | x_{k}^{i}] = \nabla f_{i} (x_{k}^{i})$ ,
$E_{ξ_{k}^{i}} [{∥ ∥ g_{i} (x_{k}^{i}, ξ_{k}^{i}) - \nabla f_{i} (x_{k}^{i}) ∥ ∥}_{i}^{2} | x_{k}^{i}] \leq σ^{2}$ .

Assumption 4 is satisfied in many scenarios, for example, when the gradient noise, $g_{i} (x_{k}^{i}, ξ_{i}) - \nabla f_{i} (x_{k}^{i})$

, is independent and identically distributed (i.i.d.) with zero-mean and finite second moment, while being independent of

x_{k}^{i}

. However, Assumption 4 allows for general gradient noise processes dependent on agent

i

and the current iterate

x_{k}^{i}

. Finally, we denote by

F_{k}

the

σ

-algebra generated by the set of random vectors

{ξ_{t}^{i}}_{0 \leq t \leq k - 1, i \in V}

Ii-a The $S$ - $A B$ algorithm

We now describe the proposed algorithm, $S$ - $A B$ , to solve Problem P1. Each agent $i$ maintains two state vectors, $x_{k}^{i}$ and $y_{k}^{i}$ , both in $R^{p}$ , where $k$ is the number of iterations. The variable $x_{k}^{i}$ is the estimate of the global minimizer $x^{*}$ , while $y_{k}^{i}$ is the global gradient estimator. The $S$ - $A B$ algorithm, initialized with arbitrary $x_{0}^{i}$ ’s and with $y_{0}^{i} = g_{i} (x_{0}^{i}, ξ_{0}^{i})$ , $\forall i \in V$ , is given by the following:


$x_{k + 1}^{i}$	$= n \sum j = 1 a_{i j} x_{k}^{j} - α y_{k}^{i},$	(2a)
$y_{k + 1}^{i}$	$= n \sum j = 1 b_{i j} y_{k}^{j} + g_{i} (x_{k + 1}^{i}, ξ_{k + 1}^{i}) - g_{i} (x_{k}^{i}, ξ_{k}^{i}),$	(2b)

where the weight matrices $A = {a_{i j}}$ and $B = {b_{i j}}$ are row- and column-stochastic, respectively, and follow the graph topology, i.e., $a_{i j} > 0$ and $b_{i j} > 0$ , iff $(i, j) \in E$ . We next write the algorithm in a compact vector form for the sake of analysis.


$x_{k + 1}$	$= A x_{k} - α y_{k},$	(3a)
$y_{k + 1}$	$= B y_{k} + g (x_{k + 1}, ξ_{k + 1}) - g (x_{k}, ξ_{k}),$	(3b)

where we use the following notation:

and $A = A \otimes I_{p}, B = B \otimes I_{p} .$

Note that when the variance, $σ$ , of the stochastic gradients is 0, we recover the $A B$ or the push-pull algorithm proposed in [22, 23]. In the following, we assume $p = 1$ for the sake of simplicity. The analysis can be extended to the general case of $p > 1$ with the help of Kronecker products.

Ii-B Auxiliary Results

We now provide some auxiliary results to aid the convergence analysis of $S$ - $A B$ . We first develop explicit norms regarding the contractions of the weight matrices, $A$ and $B$ . Since both $A$ and $B$ are primitive and stochastic, we use their non- $1_{n}$ Perron vectors, $π_{r}$ and $π_{c}$ , respectively, to define two weighted inner products as follows: $\forall x, y \in R^{n}$ ,

	$⟨ x, y ⟩_{π_{r}} ≜ x^{⊤} diag (π_{r}) y,$
	$⟨ x, y ⟩_{π_{c}} ≜ x^{⊤} diag (π_{c})^{- 1} y .$

The above inner products are well-defined because the Perron vectors, $π_{r}$ and $π_{c}$ , are positive and respectively induce a weighted Euclidean norm as follows: $\forall x \in R^{n}$ ,

	${∥ x ∥}_{π_{r}} ≜ \sqrt{[π_{r}]_{1} x_{1}^{2} + \dots + [π_{r}]_{n} x_{n}^{2}} = {∥ ∥ diag (\sqrt{π_{r}}) x ∥ ∥}_{2},$
	${∥ x ∥}_{π_{c}} ≜ \sqrt{\frac{x_{1}^{2}}{[π_{c}]_{1}} + \dots + \frac{x_{n}^{2}}{[π_{c}]_{n}}} = {∥ ∥ diag (\sqrt{π_{c}})^{- 1} x ∥ ∥}_{2}^{- 1} .$

We denote ${| | | \cdot | | |}_{π_{r}}$ and ${| | | \cdot | | |}_{π_{c}}$ as the matrix norms induced by ${∥ \cdot ∥}_{π_{r}}$ and ${∥ \cdot ∥}_{π_{c}}$ , respectively, i.e., $\forall X \in R^{n \times n}$ , see [35],

			(4)
			(5)

It can be verified that the corresponding norm equivalence relationships between $∥ \cdot ∥_{2}$ , $∥ \cdot ∥_{π_{r}}$ , and $∥ \cdot ∥_{π_{c}}$ are given by

	$∥ \cdot ∥_{π_{r}} \leq {_{r}}^{0.5} ∥ \cdot ∥_{2},$	$∥ \cdot ∥_{2} \leq {_{c}}^{0.5} ∥ \cdot ∥_{π_{c}},$
	$∥ \cdot ∥_{π_{c}} \leq {π_{c} - - -}^{- 0.5} ∥ \cdot ∥_{2},$	$∥ \cdot ∥_{2} \leq {π_{r} - - -}^{- 0.5} ∥ \cdot ∥_{π_{r}} .$

We next establish the contraction of the $A$ and $B$ matrices with the help of the above arguments.

Lemma 1

For the matrices $A, B$ , and $\forall x \in R^{n}$ , we have:

	${∥ A x - A_{\infty} x ∥}_{π_{r}} \leq σ_{A} {∥ x - A_{\infty} x ∥}_{π_{r}},$		(6)
	${∥ B x - B_{\infty} x ∥}_{π_{c}} \leq σ_{B} {∥ x - B_{\infty} x ∥}_{π_{c}},$		(7)

with $σ_{A} ≜ {| | | A - A_{\infty} | | |}_{π_{r}} < 1$ and $σ_{B} ≜ {| | | B - B_{\infty} | | |}_{π_{c}} < 1$ .

The proof of Lemma. 1 is available in the Appendix. It can be further verified that

	$σ_{A} = σ_{2} (diag (\sqrt{π_{r}}) A diag (\sqrt{π_{r}})^{- 1}),$
	$σ_{B} = σ_{2} (diag (\sqrt{π_{c}})^{- 1} B diag (\sqrt{π_{c}})),$

where $σ_{2} (\cdot)$

is the second largest singular value of a matrix.

In the following, Lemma 2 provides some simple results on the stochastic gradients, Lemma 3 uses the $l$ -smoothness of the cost functions, while Lemmas 4 and 5 are standard in convex optimization and matrix analysis. To present these results, we define three quantities:

{¯ ¯ ¯ y}_{k} ≜ \frac{1}{n} 1_{n}^{⊤} y_{k}, h (x_{k}) ≜ \frac{1}{n} 1_{n}^{⊤} \nabla f (x_{k}), {ˆ x}_{k} ≜ π_{r}^{⊤} x_{k},

where $\nabla f (x_{k}) ≜ [\nabla f_{1} (x_{k}^{1}), \dots, \nabla f_{n} (x_{k}^{n})]^{⊤} .$ The following statements use standard arguments and their formal proofs are omitted due to space limitations. Similar results can be found in [13, 22, 34].

Lemma 2

Consider $S$ - $A B$ in (2) and let Assumptions 2-4 hold. Then the following hold, $\forall k \geq 0$ :

${¯ ¯ ¯ y}_{k} = \frac{1}{n} 1_{n}^{⊤} g (x_{k}, ξ_{k}) ≜ ¯ ¯ ¯ g (x_{k}, ξ_{k})$
$E [{¯ ¯ ¯ y}_{k} | F_{k}] = \frac{1}{n} 1_{n}^{⊤} \nabla f (x_{k}) = h (x_{k})$
$E [{∥ ∥ {¯ ¯ ¯ y}_{k} - h (x_{k}) ∥ ∥}_{k}^{2} | F_{k}] \leq \frac{σ^{2}}{n}$

Lemma 3

Consider the $S$ - $A B$ algorithm in (2) and let Assumptions 2 hold. Then the following holds, $\forall k \geq 0$ :

∥ h (x_{k}) - \nabla F (x_{k}) ∥_{2} \leq \frac{l}{\sqrt{n}} ∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{2} .

Lemma 4 ([36])

Let Assumptions 1-2 hold. If $0 < α < \frac{1}{l}$ , we have: $\forall x \in R^{p}$ ,

∥ x - \nabla F (x_{k}) - x^{*} ∥_{2} \leq (1 - α μ) ∥ x - x^{*} ∥_{2} .

Lemma 5 ([35])

Let $X \in R^{n \times n}$ be non-negative and $x \in R^{n}$ be a positive vector. If $X x < ω x$ with $ω > 0$ , then $ρ (X) < ω$ .

Iii Convergence analysis

In this section, we analyze the $S$ - $A B$ algorithm and establish its convergence properties for which we present Lemmas 6-9. The proofs for these lemmas are provided in the Appendix. First, in Lemma 6, we bound $∥ y_{k} ∥_{2}^{2}$ .

Lemma 6

Let Assumptions 1-4 hold. Then the iterates ${y_{k}}_{k \geq 0}$ generated by $S$ - $A B$ in (3) follow:

	$E [∥ y_{k} ∥_{2}^{2} \| F_{k}] \leq$	$\frac{4 n ∥ π_{c} ∥_{2}^{2} l^{2}}{π_{r} - - -} ∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{π_{r}}^{2} + 4 n^{2} ∥ π_{c} ∥_{2}^{2} l^{2} ∥ {ˆ x}_{k} - x^{*} ∥_{2}^{2}$
		$+ 4_{c} E [∥ y_{k} - B_{\infty} y_{k} ∥_{π_{c}}^{2} \| F_{k}] + 4 n ∥ π_{c} ∥_{2}^{2} σ^{2} .$		(8)

Next in Lemmas 7-9, we bound the following three quantities in expectation, conditioned on the $σ$ -algebra $F_{k}$ : (i) $∥ x_{k + 1} - 1_{n} {ˆ x}_{k + 1} ∥_{π_{r}}^{2}$ , the consensus error in the network; (ii) $∥ {ˆ x}_{k + 1} - x^{*} ∥_{2}^{2}$ , the optimality gap; and, (iii) $∥ y_{k + 1} - B_{\infty} y_{k + 1} ∥_{π_{c}}^{2}$ , the gradient tracking error. We then show that the norm of a vector composed of these three quantities converges linearly to a ball around the optimal when the step-size $α$ is fixed and sufficiently small. The first lemma below is on the consensus error.

Lemma 7

Let Assumption 3 hold. Then the consensus error in the network follows:

E

[∥ x_{k + 1} - 1_{n} {ˆ x}_{k + 1} ∥_{π_{r}}^{2} | F_{k}] \leq \frac{1 + σ_{A}^{2}}{2} ∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{π_{r}}^{2} + \frac{2_{r} α^{2}}{1 - σ_{A}^{2}} E [∥ y_{k} ∥_{2}^{2} | F_{k}] .

(9)

The next lemma is on the optimality gap.

Lemma 8

Let Assumptions 1-4 hold. If $0 < α < \frac{1}{n π_{r}^{⊤} π_{c} l}$ , the optimality gap in the network follows:

$E [∥ {ˆ x}_{k + 1} - x^{*} ∥_{2}^{2} \| F_{k}] \leq$	$\frac{4 α π_{r}^{⊤} π_{c} l^{2}}{μ} ∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{π_{r}}^{2}$
	$+ (1 - (μ n π_{r}^{⊤} π_{c})^{2} α^{2}) ∥ {ˆ x}_{k} - x^{*} ∥_{2}^{2}$
	$+ \frac{2 α ∥ π_{r} ∥_{2}^{2}_{c}}{π_{r}^{⊤} π_{c} μ} E [∥ y_{k} - B_{\infty} y_{k} ∥_{π_{c}}^{2} \| F_{k}]$
	$+ 2 {(π_{r}^{⊤} π_{c})}_{c}^{2} n α^{2} σ^{2} .$	(10)

Finally, we quantify the gradient tracking error.

Lemma 9

Let the hypotheses of Lemma 2 hold. The gradient tracking error follows:

$E [∥ y_{k + 1} - B_{\infty} y_{k + 1} ∥_{π_{c}}^{2} \| F_{k}] \leq$	$\frac{1 + σ_{B}^{2}}{2} E [∥ y_{k} - B_{\infty} y_{k} ∥_{π_{c}}^{2} \| F_{k}]$
	$+ \frac{32 l^{2}}{π_{r} - - - π_{c} - - - (1 - σ_{B}^{2})} ∥ x_{k} - 1_{n} {¯ ¯ ¯ x}_{k} ∥_{π_{r}}^{2}$
	$+ \frac{4 l^{2} α^{2}}{π_{r} - - - π_{c} - - - (1 - σ_{B}^{2})} E [∥ y_{k} ∥_{2}^{2} \| F_{k}] + \frac{8 n σ^{2}}{- π_{c}} .$	(11)

With the help of the above lemmas, we define a vector, $t_{k} \in R^{3}$ , i.e.,

t_{k} = ⎡ ⎢ ⎢ ⎣ \begin{matrix} E [∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{π_{r}}^{2}] E [∥ {ˆ x}_{k} - x^{*} ∥_{2}^{2}] E [∥ y_{k} - B_{\infty} y_{k} ∥_{π_{c}}^{2}] . \end{matrix} ⎤ ⎥ ⎥ ⎦

By substituting the bound on $E [∥ y_{k} ∥_{2}^{2} | F_{k}]$ from Lemma 6 in Lemmas 7-9, and taking the full expectation of both sides, it can be verified that $t_{k}$ follows the dynamical system below.

	$t_{k + 1} \leq$	$⎡ ⎢ ⎢ ⎢ ⎢ ⎣ \begin{matrix} \frac{1 + σ_{A}^{2}}{2} + a_{1} α^{2} & a_{2} α^{2} & a_{3} α^{2} a_{4} α & 1 - a_{5} α^{2} & a_{6} α a_{7} + a_{8} α^{2} & a_{9} α^{2} & \frac{1 + σ_{B}^{2}}{2} + a_{10} α^{2} \end{matrix} ⎤ ⎥ ⎥ ⎥ ⎥ ⎦ t_{k} + ⎡ ⎢ ⎣ \begin{matrix} b_{1} α^{2} b_{2} α^{2} b_{3} + b_{4} α^{2} \end{matrix} ⎤ ⎥ ⎦,$
	$≜$	$G_{α} t_{k} + b_{α},$		(12)

where the constants are given by

\begin{matrix} a_{1} = \frac{8 n ∥ π_{c} ∥_{2}^{2} l^{2}}{1 - σ_{A}^{2}}, & a_{8} = \frac{16 n l^{4} ∥ π_{c} ∥_{2}^{2}}{{_{r}}^{2}_{c} (1 - σ_{B}^{2})} a_{2} = \frac{8 n ∥ π_{c} ∥_{2}^{2} l^{2}_{r}}{1 - σ_{A}^{2}}, & a_{9} = \frac{16 n^{2} l^{4} ∥ π_{c} ∥_{2}^{2}}{_{r}_{c} (1 - σ_{B}^{2})}, a_{3} = \frac{8_{r}_{c}}{1 - σ_{A}^{2}}, & a_{10} = \frac{16 l^{2}_{c}}{π_{r} - - - π_{c} - - -}, a_{4} = \frac{4 π_{r}^{⊤} π_{c} l^{2}}{μ}, & b_{1} = \frac{8 n ∥ π_{c} ∥_{2}^{2}_{r} σ^{2}}{1 - σ_{A}^{2}}, a_{5} = (μ n π_{r}^{⊤} π_{c})^{2}, & b_{2} = 2 (π_{r}^{⊤} π_{c})^{2} n σ^{2}, a_{6} = \frac{2 ∥ π_{r} ∥_{2}^{2}_{c}}{} π_{r}^{⊤} π_{c} μ, & b_{3} = \frac{8 n σ^{2}}{π_{c} - - -}, a_{7} = \frac{32 l^{2}}{π_{c} - - - π_{r} - - - (1 - σ_{B}^{2})}, & b_{4} = \frac{16 n l^{2} ∥ π_{c} ∥_{2}^{2} σ^{2}}{π_{r} - - - π_{c} - - - (1 - σ_{B}^{2})} . \end{matrix}

Iv Main Result

In this section, we analyze the inequality on $t_{k}$ to establish the convergence of $S$ - $A B$ .

Theorem 1

Consider the $S$ - $A B$ algorithm in (2) and let Assumptions 1-4 hold. Suppose the step-size $α$ satisfies the following the condition:

0 < α < min ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ \frac{1}{n π_{r}^{⊤} π_{c} l}, \frac{1 - σ_{A}^{2}}{\frac{2 a_{1}}{l} + \frac{4 a_{2}}{a_{5}} (a_{4} + \frac{4 a_{6} a_{7}}{1 - σ_{B}^{2}}) + \frac{8 a_{3} a_{7}}{l (1 - σ_{B}^{2})}}, \frac{a_{7}}{\frac{a_{8}}{l} + \frac{2 a_{9}}{a_{5}} (a_{4} + \frac{4 a_{6} a_{7}}{1 - σ_{B}^{2}}) + \frac{4 a_{7} a_{10}}{l (1 - σ_{B}^{2})}} ⎫ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎭ .

Then, $ρ (G_{α}) < 1$ , the vector $(I_{3} - G_{α})^{- 1} b_{α}$ has non-negative components, and we have that

	$lim sup k \to \infty t_{k} \leq (I_{3} - G_{α})^{- 1} b_{α}$
	$lim sup k \to \infty E [∥ x_{k} - 1_{n} {ˆ x}_{k} ∥_{π_{r}}^{2}] \leq {[(I_{3} - G_{α})^{- 1} b_{α}]}_{1}^{- 1}$

where the above convergence is geometric with exponent $ρ (G_{α})$ .

The goal is to find the range of $α$ such that $ρ (G_{α}) < 1$ . In the light of Lemma 5, it suffices to solve for the range of $α$ such that $G_{α} δ < δ$ holds for some positive vector $δ = [δ_{1}, δ_{2}, δ_{3}]^{⊤}$ . We now expand this element-wise matrix inequality as follows:

	$(\frac{1 + σ_{A}^{2}}{2} + a_{1} α^{2}) δ_{1} + a_{2} α^{2} δ_{2} + a_{3} α^{2} δ_{3}$	$< δ_{1}$
	$a_{4} α δ_{1} + (1 - a_{5} α^{2}) δ_{2} + a_{6} α δ_{3}$	$< δ_{2}$
	$(a_{7} + a_{8} α^{2}) δ_{1} + a_{9} α^{2} δ_{2} + (\frac{1 + σ_{B}^{2}}{2} + a_{10} α^{2}) δ_{3}$	$< δ_{3}$

which can be reformulated as follows:

$(a_{1} δ_{1} + a_{2} δ_{2} + a_{3} δ_{3}) α^{2}$	$< \frac{1 - σ_{A}^{2}}{2} δ_{1}$	(13)
$a_{4} δ_{1} α - a_{5} δ_{2} α^{2} + a_{6} δ_{3} α$	$< 0$	(14)
$(a_{8} δ_{1} + a_{9} δ_{2} + a_{10} δ_{3}) α^{2}$	$< \frac{1 - σ_{B}^{2}}{2} δ_{3} - a_{7} δ_{1} .$	(15)

We now set $δ_{1}, δ_{2}, δ_{3}$ as

δ_{1} = α, δ_{2} = \frac{2}{a_{5}} (a_{4} + \frac{4 a_{6} a_{7}}{1 - σ_{B}^{2}}), δ_{3} = \frac{4 a_{7}}{1 - σ_{B}^{2}} α .

(16)

Then for (13) to hold, it suffices to require

α < \frac{1 - σ_{A}^{2}}{\frac{2 a_{1}}{l} + \frac{4 a_{2}}{a_{5}} (a_{4} + \frac{4 a_{6} a_{7}}{1 - σ_{B}^{2}}) + \frac{8 a_{3} a_{7}}{l (1 - σ_{B}^{2})}} .

(17)

One can verify that with the choices of $δ_{1}, δ_{2}, δ_{3}$ provided in (16), (14) holds. Lastly, for (15) to hold, we have

α < \frac{a_{7}}{\frac{a_{8}}{l} + \frac{2 a_{9}}{a_{5}} (a_{4} + \frac{4 a_{6} a_{7}}{1 - σ_{B}^{2}}) + \frac{4 a_{7} a_{10}}{l (1 - σ_{B}^{2})}} .

(18)

Therefore, (17) and (18) together with the requirement that $α < \frac{1}{n π_{r}^{⊤} π_{c} l}$ from Lemma 8 complete the proof. It is important to note that the error bounds in Theorem 1 go to zero as the step-size gets smaller and the variance on the gradient noise decreases.

V Numerical Experiments

In this section, we illustrate the $S$ - $A B$ algorithm and its convergence properties. We demonstrate the results on a directed graph generated using nearest neighbor rules with $n = 20$ agents. The particular graph for the experiments is shown in Fig. 1

(left) to provide a sense of connectivity. We choose a logistic regression problem to classify around

12, 000

images of two digits,

7

and

8

, labeled as

y_{i j} = + 1

- 1

, from the MNIST dataset [37]. Each image,

c_{i j}

, is a

785

-dimensional vector and the total images are divided among the agents such that each agent has

m_{i} = 600

images. Because privacy and communication restrictions, the agents do not share their local batches (local training images) with each other. In order to use the entire data set for training, the network of agents cooperatively solves the following distributed logistic regression problem:

min w, b F (w, b) = n \sum i = 1 m_{i} \sum j = 1 l n [1 + e^{- (w^{⊤} c_{i j} + b) y_{i j}}] + \frac{λ}{2} ∥ w ∥_{2}^{2},

where the private function at each agent, $i$ , is given by:

f_{i} (w, b) = m_{i} \sum j = 1 l n [1 + e^{- (w^{⊤} c_{i j} + b) y_{i j}}] + \frac{λ}{2 n} ∥ w ∥_{2}^{2} .

We show the performance of this classification problem over centralized and distributed methods. Centralized gradient descent (CGD) uses the entire batch, i.e., it computes $12, 000$

gradients at each iteration, whereas centralized stochastic gradient descent (C-SGD) uses only one data point at each iteration that is uniformly sampled from the entire batch. For the distributed algorithms, we show the performance of non-stochastic

A B

, where each agent uses its entire local batch, i.e.,

600

labeled data points. Whereas, for the implementation of

S

A B

, each agent uniformly chooses one data point from its local batch. For testing, we use

2000

additional images that were not used for training. The residuals are shown in Fig. 1 (right) while the training and testing accuracy is shown in Fig. 2

. In the performance figures, the horizontal axis represents the number of epochs where each epoch represents computations on the entire batch. Clearly,

S

A B

has a better performance when compared to

A B

in [22] as expected from the performance of their centralized counterparts, C-SGD and CGD.

Fig. 1: (Left) Strongly-connected directed graph. (Right) Residuals.

Fig. 2: (Left) Training accuracy. (Right) Test accuracy.

Vi Conclusions

In this paper, we have presented a stochastic gradient descent algorithm, $S$ - $A B$ , over arbitrary strongly-connected graphs. In this setup, the data is distributed over agents and each agent uniformly samples a data point (from its local batch) at each iteration of the algorithm to implement the stochastic $S$ - $A B$ algorithm. To cope with general directed communication graphs and potential lack of doubly-stochastic weights, $S$ - $A B$ employs a two-phase update with row- and column-stochastic weights. We have shown that under a sufficiently small constant step-size, $S$ - $A B$ converges linearly to a neighborhood of the global minimizer when the local cost functions are smooth and strongly-convex. We have presented numerical simulations based on real-world datasets to illustrate the theoretical results.

References

[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics, 2017, pp. 1273–1282.
[2] H. Raja and W. U. Bajwa, “Cloud K-SVD: A collaborative dictionary learing algorithm for big, distributed data,” IEEE Trans. Signal Processing, vol. 64, no. 1, pp. 173–188, Jan. 2016.
[3] A. Daneshmand, F. Facchinei, V. Kungurtsev, and G. Scutari, “Hybrid random/deterministic parallel algorithms for convex and nonconvex big data optimization,” IEEE Trans. on Signal Processing, vol. 63, no. 15, pp. 3914–3929, 2015.
[4] D. Jakovetić, J. Xavier, and J. M. F. Moura, “Fast distributed gradient methods,” IEEE Trans. on Automatic Control, vol. 59, no. 5, pp. 1131–1146, May 2014.
[5] F. Bullo, J. Cortés, and S. Martinez, Distributed control of robotic networks: A mathematical approach to motion coordination algorithms, Princeton University Press, 2009.
[6] S. Lee and M. M. Zavlanos, “Approximate projection methods for decentralized optimization with functional constraints,” IEEE Trans. on Automatic Control, 2017.
[7] A. Nedić and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. on Automatic Control, vol. 54, no. 1, pp. 48–61, Jan. 2009.
[8] I. Lobel, A. Ozdaglar, and D. Feijer, “Distributed multi-agent optimization with state-dependent communication,” Mathematical Programming, vol. 129, no. 2, pp. 255–284, 2011.
[9] J. C. Duchi, A. Agarwal, and M. J. Wainwright, “Dual averaging for distributed optimization: Convergence analysis and network scaling,” IEEE Trans. on Automatic Control, vol. 57, no. 3, pp. 592–606, Mar. 2012.
[10] S. S. Ram, A. Nedić, and V. V. Veeravalli, “Distributed stochastic subgradient projection algorithms for convex optimization,” Journal of optimization theory and applications, vol. 147, no. 3, pp. 516–545, 2010.
[11] A. Nedić and A. Olshevsky, “Stochastic gradient-push for strongly convex functions on time-varying directed graphs,” IEEE Trans. on Automatic Control, vol. 61, no. 12, pp. 3936–3947, Dec. 2016.
[12] J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,” in IEEE 54th Annual Conference on Decision and Control, 2015, pp. 2055–2060.
[13] G. Qu and N. Li, “Harnessing smoothness to accelerate distributed optimization,” IEEE Trans. on Control of Network Systems, Apr. 2017.
[14] A. Nedić, A. Olshevsky, and W. Shi, “Achieving geometric convergence for distributed optimization over time-varying graphs,” SIAM Journal of Optimization, vol. 27, no. 4, pp. 2597–2633, Dec. 2017.
[15] C. Xi, R. Xin, and U. A. Khan, “ADD-OPT: Accelerated distributed directed optimization,” IEEE Trans. on Automatic Control, Aug. 2017, in press.
[16] K. I. Tsianos, S. Lawlor, and M. G. Rabbat, “Push-sum distributed dual averaging for convex optimization,” in 51st IEEE Annual Conference on Decision and Control, Maui, Hawaii, Dec. 2012, pp. 5453–5458.
[17] A. Nedić and A. Olshevsky, “Distributed optimization over time-varying directed graphs,” IEEE Trans. on Automatic Control, vol. 60, no. 3, pp. 601–615, Mar. 2015.
[18] C. Xi, Q. Wu, and U. A. Khan, “On the distributed optimization over directed networks,” Neurocomputing, vol. 267, pp. 508–515, Dec. 2017.
[19] C. Xi and U. A. Khan, “Distributed subgradient projection algorithm over directed graphs,” IEEE Trans. on Automatic Control, vol. 62, no. 8, pp. 3986–3992, Oct. 2016.
[20] C. Xi, V. S. Mai, R. Xin, E. Abed, and U. A. Khan, “Linear convergence in optimization over directed graphs with row-stochastic matrices,” IEEE Trans. on Automatic Control, Jan. 2018, in press.
[21] R. Xin, C. Xi, and U. A. Khan, “FROST – Fast row-stochastic optimization with uncoordinated step-sizes,” Arxiv: https://arxiv.org/abs/1803.09169, Mar. 2018.
[22] R. Xin and U. A. Khan, “A linear algorithm for optimization over directed graphs with geometric convergence,” IEEE Control Systems Letters, vol. 2, no. 3, pp. 325–330, Jul. 2018.
[23] S. Pu, W. Shi, J. Xu, and A. Nedić, “A push-pull gradient method for distributed optimization in networks,” in 57th IEEE Annual Conference on Decision and Control, Dec. 2018.
[24] R. Xin and U. A. Khan, “Distributed heavy-ball: A generalization and acceleration of first-order methods with gradient tracking,” arXiv preprint arXiv:1808.02942, 2018.
[25] S. Pu, W. Shi, J. Xu, and A. Nedić, “Push-pull gradient methods for distributed optimization in networks,” https://arxiv.org/abs/1810.06653, 2018.
[26] F. Saadatniaki, R. Xin, and U. A. Khan, “Optimization over time-varying directed graphs with row and column-stochastic matrices,” arXiv preprint arXiv:1810.07393, 2018.
[27] A. Daneshmand, G. Scutari, and V. Kungurtsev, “Second-order guarantees of distributed gradient algorithms,” arXiv preprint arXiv:1809.08694, 2018.
[28] R. Xin, D. Jakovetic, and U. A. Khan, “Distributed Nesterov gradient methods over arbitrary graphs,” IEEE Signal Processing Letters, Jan. 2019, Arxiv: 1901.06995.
[29] D. Yuan, Y. Hong, D. W. C. Ho, and G. Jiang, “Optimal distributed stochastic mirror descent for strongly convex optimization,” Automatica, vol. 90, pp. 196–203, Apr. 2018.
[30] N. Denizcan Vanli, Muhammed O. Sayin, and Suleyman S. Kozat, “Stochastic subgradient algorithms for strongly convex optimization over distributed networks,” IEEE Trans. on Network Science and Engineering, vol. 4, no. 4, pp. 248–260, Oct. 2017.
[31] D. Jakovetić, D. Bajović, A. K. Sahu, and S. Kar, “Convergence rates for distributed stochastic optimization over random networks,” in IEEE Conference on Decision and Control, Dec. 2018, pp. 4238–4245.
[32] A. K. Sahu, D. Jakovetić, D. Bajović, and S. Kar, “Distributed zeroth order optimization over random networks: A Kiefer-Wolfowitz stochastic approximation approach,” in Conference on Decision and Control, Dec. 2018, pp. 4951–4958.
[33] D. Jakovetic, “A unification and generalization of exact distributed first order methods,” IEEE Trans. on Signal and Information Processing over Networks, 2018.
[34] S. Pu and A. Nedić, “A distributed stochastic gradient tracking method,” in 2018 IEEE Conference on Decision and Control (CDC), Dec. 2018, pp. 963–968.
[35] R. A. Horn and C. R. Johnson, Matrix Analysis, 2 $\scriptsize nd$ ed., Cambridge University Press, New York, NY, 2013.
[36] Y. Nesterov, Introductory lectures on convex optimization: A basic course, vol. 87, Springer Science & Business Media, 2013.
[37] Y. LeCun, C. Cortes, and C. Burges, “MNIST handwritten digit database,” AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist, vol. 2, pp. 18, 2010.

Proof of Lemma 1

We start with the proof of (6). Note that $A x - A_{\infty} x = (A - A_{\infty}) (x - A_{\infty} x)$ that leads to

By the definition of ${| | | \cdot | | |}_{π_{r}}$ and Eq. (4), we have

{| | | A - A_{\infty} | | |}_{π_{r}}

= {∣ ∣ ∣ ∣ ∣ ∣ % diag (\sqrt{π_{r}}) (A - A_{\infty}) diag (\sqrt{π_{r}})^{- 1} ∣ ∣ ∣ ∣ ∣ ∣}_{2} ≜ \sqrt{λ_{max} (J)},

where $λ_{max} (\cdot)$ denotes the largest eigenvalue of the matrix and $J = diag (\sqrt{π_{r}})^{- 1} (A - A_{\infty})^{⊤} diag% (π_{r}) (A - A_{\infty}) diag (\sqrt{π_{r}})^{- 1}$ . What we need to show is that $ρ (J) < 1$ . Expanding $J$ , we get

	$J =$	$diag (\sqrt{π_{r}})^{- 1} A^{⊤} % diag (π_{r}) A diag (\sqrt{π_{r}})^{- 1}$
		$- diag (\sqrt{π_{r}})^{- 1} A_{\infty}^{⊤} diag (π_{r}) A diag (\sqrt{π_{r}})^{- 1}$
		$- diag (\sqrt{π_{r}})^{- 1} A^{⊤} diag (π_{r}) A_{\infty} diag (\sqrt{π_{r}})^{- 1}$
		$+ diag (\sqrt{π_{r}})^{- 1} A_{\infty}^{⊤} diag (π_{r}) A_{\infty} diag (\sqrt{π_{r}})^{- 1},$
	$≜$	$J_{1} - J_{2} - J_{3} + J_{4} .$

With the fact that $A_{\infty} = 1_{n} π_{r}^{⊤}$ , it can be verified that $J_{2} = J_{3} = J_{4} = {\sqrt{π}}_{r} {\sqrt{π}}_{r}^{⊤}$ and thus $J = J_{1} - {\sqrt{π}}_{r} {\sqrt{π}}_{r}^{⊤} .$ Furthermore, $J_{1} \sqrt{π_{r}} = \sqrt{π_{r}}$ , and ${\sqrt{π_{r}}}^{⊤} J_{1} = {\sqrt{π_{r}}}^{⊤} .$ Since $J_{1}$ is primitive, by Perron-Frobenius theorem [35], we have $ρ (J) = ρ (J_{1} - {\sqrt{π}}$

$E [∥ y_{k + 1} - B_{\infty} y_{k + 1} ∥_{π_{c}}^{2} \| F_{k}] \leq$	$\frac{1 + σ_{B}^{2}}{2} E [∥ y_{k} - B_{\infty} y_{k} ∥_{π_{c}}^{2} \| F_{k}]$
	$+ \frac{32 l^{2}}{π_{r} - - - π_{c} - - - (1 - σ_{B}^{2})} ∥ x_{k} - 1_{n} {¯ ¯ ¯ x}_{k} ∥_{π_{r}}^{2}$
	$+ \frac{4 l^{2} α^{2}}{π_{r} - - - π_{c} - - - (1 - σ_{B}^{2})} E [∥ y_{k} ∥_{2}^{2} \| F_{k}] + \frac{8 n σ^{2}}{- π_{c}} .$	(11)

Distributed stochastic optimization with gradient tracking over strongly-connected networks

Authors

Variance reduced stochastic optimization over directed graphs with row and column stochastic weights

Distributed Nesterov gradient methods over arbitrary graphs

S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs

Linearly Convergent Algorithm with Variance Reduction for Distributed Stochastic Optimization

Distributed Optimization over Directed Graphs with Row Stochasticity and Constraint Regularity

Three-Dimensional Swarming Using Cyclic Stochastic Optimization

FADE: Fast and Asymptotically efficient Distributed Estimator for dynamic networks

I Introduction

Ii Problem formulation and Auxiliary Results

Assumption 1

Assumption 2

Assumption 3

Assumption 4

Ii-a The $S$ - $A B$ algorithm

Ii-B Auxiliary Results

Lemma 1

Lemma 2

Lemma 3

Lemma 4 ([36])

Lemma 5 ([35])

Iii Convergence analysis

Lemma 6

Lemma 7

Lemma 8

Lemma 9

Iv Main Result

Theorem 1

V Numerical Experiments

Vi Conclusions

References

Proof of Lemma 1

Distributed stochastic optimization with gradient tracking over strongly-connected networks

Authors

Related Research

Variance reduced stochastic optimization over directed graphs with row and column stochastic weights

Distributed Nesterov gradient methods over arbitrary graphs

S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs

Linearly Convergent Algorithm with Variance Reduction for Distributed Stochastic Optimization

Distributed Optimization over Directed Graphs with Row Stochasticity and Constraint Regularity

Three-Dimensional Swarming Using Cyclic Stochastic Optimization

FADE: Fast and Asymptotically efficient Distributed Estimator for dynamic networks

This week in AI

I Introduction

Ii Problem formulation and Auxiliary Results

Assumption 1

Assumption 2

Assumption 3

Assumption 4

Ii-a The S-AB algorithm

Ii-B Auxiliary Results

Lemma 1

Lemma 2

Lemma 3

Lemma 4 ([36])

Lemma 5 ([35])

Iii Convergence analysis

Lemma 6

Lemma 7

Lemma 8

Lemma 9

Iv Main Result

Theorem 1

V Numerical Experiments

Vi Conclusions

References

Proof of Lemma 1

Ii-a The $S$ - $A B$ algorithm