Forward-backward-forward methods with variance reduction for stochastic variational inequalities

1. Introduction

In this paper we consider the following variational inequality problem, denoted as $VI (T, X)$ , or simply $VI$ : given a nonempty closed and convex set $X \subseteq R^{d}$ and a single valued map $T : R^{d} \to R^{d}$ , find $x^{*} \in X$ such that

⟨ T (x^{*}), x - x^{*} ⟩ \geq 0 for all x \in X .

(1.1)

We call $S (T, X) \equiv X_{*}$ the set of (Stampacchia) solutions of $VI (T, X)$ . The variational inequality problem (1.1) arises in many interesting applications in economics, game theory and engineering [29, 27, 19, 18, 24], and includes as a special case first-order optimality conditions for nonlinear optimization, by choosing $T = \nabla f$ for some smooth function $f$ . If $X$ is unbounded, it can also be used to formulate complementarity problems, systems of equations, saddle point problems and many equilibrium problems. We refer the reader to [11] for an extensive review of applications in engineering and economics.

In many instances the problem $VI$

arises as the expected value of an underlying stochastic optimization problem whose primitives are defined on a probability space

(Ω, F, P)

carrying a random variable

ξ : (Ω, F) \to (Ξ, A)

taking values in a measurable space

(Ξ, A)

and inducing a law

P = P \circ ξ^{- 1}

. Given the random element

ξ

, consider the measurable mapping

F : X \times Ξ \to R^{d}

, defining an integrable random vector

F (x, ξ) : Ω \to R^{d}

via the composition

F (x, ξ) (ω) = F (x, ξ (ω))

. The stochastic variational inequality problem on which we will focus in this paper is denoted by

SVI

and defined as follows:

Definition 1.1.

Let the operator $T : R^{d} \to R^{d}$ be defined by

T (x) := E_{ξ} [F (x, ξ)] := \int_{Ω} F (x, ξ (ω)) d P (ω) = \int_{Ξ} F (x, z) d P (z) .

(1.2)

Find $x^{*} \in X$ satisfying (1.1).

This definition is known as the expected value formulation of the stochastic variational inequality problem. The expected value formulation goes back to the seminal work of [20]. By its very definition, if the operator $T$ defined in (1.2) would be known, then the expected value formulation can be solved by any standard solution technique for deterministic variational inequalities. However, in practice, the operator $T$ is usually not directly accessible, either due to excessive computations involved in performing the integral, or because $T$ itself is the solution of an embedded subproblem. Hence, in most situations of interest, the solution of $SVI$ relies on random samples of the operator $F (x, ξ)$ . In this context, there are two current methodologies available; the sample average approximation (SAA

) approach replaces the expected value formulation with an empirical estimator of the form

{^T}^{N} (x) = \frac{1}{N} N \sum j = 1 F (x, ξ_{j}),

and use the resulting deterministic map $T^{N}$ as the input in one existing algorithm of choice. We refer to [30] for this solution approach in connection with Monte Carlo simulation. We note that this approach is the standard choice in expected residual minimization problems, when $P$ is unknown but accessible via a Monte Carlo approach.

A different methodology is the stochastic approximation (SA) approach, where samples are obtained in an online fashion, namely, the decision maker chooses one deterministic algorithm to solve the expected value formulation, and draws a fresh random variable whenever needed. The mechanism to draw a fresh sample from $P$ is usually named a stochastic oracle (SO), which report generates a stochastic error $F (x, ξ) - T (x)$ .

Until very recently, the SA approach has only been used for the expected value formulation under very restrictive assumptions. To the best of our knowledge, the first formulation of an SA approach for a stochastic VI problem was made by [16], under the assumption of strong monotonicity and continuity of the operator $T$ . There, a proximal point algorithm of the form

X_{n + 1} = Π_{X} [X_{n} + α_{n} F (x_{n}, ξ_{n})]

(1.3)

is considered, where $Π_{X}$ denotes the Euclidean projection onto $X$ , $(ξ_{n})_{n \geq 0}$ is a sample of $P$ , and $(α_{n})_{n \geq 0}$ is a sequence of positive step sizes. Almost sure convergence of the iterates is proven for small step sizes, assuming $T$ is Lipschitz continuous and strongly monotone, and the stochastic error is uniformly bounded. Relaxing strong monotonicity to plain monotonicity, the recent paper [37] incorporated a Tikhonov regularization scheme into the stochastic approximation algorithm (1.3) and proved almost sure convergence of the generated stochastic process. The only established method guaranteeing almost sure convergence under the significantly weaker assumption of pseudo-monotonicty of the mean operator is the extragradient approach of [15]. The original Korpelevich extragradient scheme of [21] consists of two projection steps using two evaluations of the deterministic map $T$ at generated test points $y_{n}$ and $x_{n}$ . Extending this to the stochastic oracle case, we arrive at the stochastic extra-gradient (SEG) method

	$Y_{n}$	$= Π_{X} [X_{n} - α_{n} A_{n + 1}]$		(SEG)
	$X_{n + 1}$	$= Π_{X} [X_{n} - α_{n} B_{n + 1}]$		(SEG)

where $(A_{n})_{n \geq 1}, (B_{n})_{n \geq 1}$ are stochastic estimators of $T (X_{n})$ , and $T (Y_{n})$ , respectively. The paper [15] constructs these estimators by relying on a dynamic sampling strategy, where noise reduction of the estimators is achieved via a mini-batch sampling of the stochastic operators $F (X_{n}, ξ$ ) and $F (Y_{n}, ξ)$ . Within this mini-batch formulation, almost sure convergence of the stochastic process $(X_{n})_{n \in N}$ to the solution set can be proven even with constant step size implementations of SEG. On top, optimal convergence rates of $O (1 / N)$ in terms of the mean squared residual of the $VI$ are obtained.

1.1. Our Contribution

We briefly summarize the main contributions of this work. The most costly part of SEG are the two separate projection steps performed at each single iteration of the method. We show in this paper that a stochastic version of Tseng’s forward-backward-forward [35], which we call the stochastic forward-backward-forward (SFBF) algorithm, preserves the strong trajectory-based convergence results, while the saving of one projection step allows us to beat SEG significantly in terms of computational overheads and runtimes. In terms of convergence properties the SFBF algorithm developed in this paper has the same good properties as SEG. However, SFBF is potentially more efficient than SEG in each iteration since it relies only on a single euclidean projection step. The price to pay for this is that we obtain an infeasible method (as is typical for primal-dual schemes) with a lower computational complexity count at the positive side. Additionally, the theoretically allowed range for step sizes is by the constant factor $\sqrt{3}$ times larger than the theoretically allowed largest step size in SEG. This constant factor gain results in significant improvements in terms of the convergence speed. This will be illustrated with extensive numerical evidences reported in Section 6.

2. Preliminaries

2.1. Notation

For $x, y \in R^{d}$ , we denote by $⟨ x, y ⟩$ the standard inner product, and by $∥ x ∥ \equiv ∥ x ∥_{2} := ⟨ x, x ⟩^{\frac{1}{2}}$ the corresponding norm. For $p \in [1, \infty]$ , the $ℓ_{p}$ norm on $R^{d}$ is defined for $x = (x_{1}, . . ., x_{p})$ as $∥ x ∥_{p} := {(\sum_{i = 1}^{n} | x_{i} |^{p})}_{i}^{\frac{1}{p}}$ . For a nonempty, closed and convex set $E \subseteq R^{d}$ , the Euclidean projector is defined as $Π_{E} (x) := {argmin}_{y \in E} ∥ y - x ∥$ for $x \in R^{d}$ . All random elements are defined on a given probability space $(Ω, F, P)$ . An $E$ -valued random variable is a $(F, E)$ -measurable mapping $f : Ω \to E$ ; we write $f \in L^{0} (Ω, F, P; E)$ . For every $p \in [1, \infty]$ , define the equivalence class of random variables $f \in L^{0} (Ω, F, P; E)$ with $E (∥ f ∥^{p})^{1 / p} < \infty$ as $L^{p} (Ω, F, P; E)$ . If $G \subseteq F$ , the conditional expectation of the random variable $f \in L^{p} (Ω, F, P; E)$ is denoted by $E [f | G]$ . For $f_{1}, \dots, f_{k} \in L^{p} (Ω, F, P; E)$ we denote the sigma-algebra generated by these random variables by $σ (f_{1}, \dots, f_{k})$ , this is the smallest sigma-algebra measuring the random variables $f_{1}, \dots, f_{k}$ . Let $(Ω, F, F = (F_{n})_{n \geq 0}, P)$ be a complete stochastic basis. We denote by $ℓ^{0} (F)$ the set of random sequences $(ξ_{n})_{n \geq 1}$ such for each $n \in N$ , $ξ_{n} \in L^{0} (Ω, F_{n}, P; R)$ . For $p \in [1, \infty]$ , we set

ℓ^{p} (F) ≜ {(ξ_{n})_{n \geq 1} \in ℓ^{0} (F) | \sum n \geq 1 | ξ_{n} |^{p} < \infty P % - a . s .} .

The following properties of the euclidean projection on a closed and convex set are well known.

Lemma 2.1.

Let $K \subseteq R^{d}$ be a nonempty, closed and convex set. Then:

$Π_{K} (x)$ is the unique point of $K$ satisfying $⟨ x - Π_{K} (x), y - Π_{K} (x) ⟩ \leq 0$ for all $y \in K$ ;
for all $x \in R^{d}$ and $y \in K$ , we have $∥ Π_{K} (x) - y ∥^{2} + ∥ Π_{K} (x) - x ∥^{2} \leq ∥ x - y ∥^{2}$ ;
for all $x, y \in R^{d}$ , $∥ Π_{K} (x) - Π_{K} (y) ∥ \leq ∥ x - y ∥$ ;
given $α > 0$ and $T : K \to R^{d}$ , the set of solutions of the variational problem $VI (T, K)$ can be expressed as $S (T, K) = {x \in R^{d} | x = Π_{K} (x - α T (x))}$ .

Remark 2.1.

In the literature on variational inequalities, there exists an alternative solution concept known as weak, or Minty, solutions. In this paper we are only interested in strong, or Stampacchia, solutions of $VI (T, K)$ , defined by inequality (1.1).

Another useful fact we use in this paper is the following elementary identity.

Lemma 2.2 (Pythagorean identity).

For all $x, x_{n}, x_{n + 1} \in R^{d}$ we have

∥ x_{n + 1} - x ∥^{2} + ∥ x_{n + 1} - x_{n} ∥^{2} - ∥ x_{n} - x ∥^{2} = 2 ⟨ x_{n + 1} - x_{n}, x_{n + 1} - x ⟩ .

2.2. Probabilistic Tools

We recall the Minkowski inequality: for given functions $f, g \in L^{p} (Ω, F, P; E), G \subseteq F$ and $p \in [1, \infty]$ , we have

E [∥ f + g ∥^{p} | G]^{1 / p} \leq E [∥ f ∥^{p} | G]^{1 / p} + E [∥ g ∥^{p} | G]^{1 / p} .

(2.1)

For the convergence analysis we will make use of the following classical lemma (see e.g. [26, Lemma 11, page 50]).

Lemma 2.3 (Robbins-Siegmund).

Let $(Ω, F, F = (F_{n})_{n \geq 0}, P)$ be a discrete stochastic basis. Let $(v_{n})_{n \geq 1}, (u_{n})_{n \geq 1} \in ℓ_{+}^{0} (F)$ and $(θ_{n})_{n \geq 1}, (β_{n})_{n \geq 1} \in ℓ_{+}^{1} (F)$ be such that for all $n \geq 0$

E [v_{n + 1} | F_{n}] \leq (1 + θ_{n}) v_{n} - u_{n} + β_{n} P - a.s. .

Then $(v_{n})_{n \geq 0}$ converges a.s. to a random variable $v$ , and $(u_{n})_{n \geq 1} \in ℓ_{+}^{1} (F)$ .

Finally, we need the celebrated Burkholder-Davis-Gundy inequality (see e.g. [33]).

Lemma 2.4.

Let $(Ω, F, (F_{n})_{n \geq 0}, P)$ be a discrete stochastic basis and $(U_{n})_{n \geq 0}$ a vector-valued martingale relative to this basis. Then, for all $p \in [1, \infty)$ , there exists a universal constant $C_{p} > 0$ such that for every $N \geq 1$

E {[{(sup 0 \leq i \leq N ∥ U_{i} ∥)}_{i}^{p}]}^{1 / p} \leq C_{p} E {⎡ ⎢ ⎣ {(N \sum i = 1 ∥ U_{i} - U_{i - 1} ∥^{2})}_{i}^{p / 2} ⎤ ⎥ ⎦}^{\frac{1}{p}} .

When combined with Minkowski inequality, we obtain for all $p \geq 2$ a constant $C_{p} > 0$ such that for every $N \geq 1$

E {[{(sup 0 \leq i \leq N ∥ U_{i} ∥)}_{i}^{p}]}^{1 / p} \leq C_{p} \sqrt{N \sum k = 1 E {(∥ U_{i} - U_{i - 1} ∥^{p})}_{i}^{2 / p}}

3. The stochastic forward-backward-forward algorithm

In this paper we study a forward-backward-forward algorithm of Tseng type under weak monotonicity assumptions. The blanket hypotheses we consider throughout our analysis are summarized here:

Assumption 1 (Consistency).

The solution set $X_{*} \equiv S (T, X)$ is nonemtpy.

Assumption 2 (Stochastic Model).

The set $X \subseteq R^{d}$ is nonempty, closed and convex, $(Ξ, A)$ is a measurable space and $F : R^{d} \times Ξ \to R^{d}$ is a Carathéodory map.¹¹1The mapping $x \mapsto F (x, ξ)$ is continuous for a. e. $ξ \in Ξ$ , and $ξ \mapsto F (x, ξ)$ is measurable for all $x \in R^{d}$ ; $ξ$ is a random variable with values in $Ξ$ , defined on a probability space $(Ω, F, P)$ .

Assumption 3 (Lipschitz continuity).

The averaged operator $T (\cdot) = E_{ξ} [F (\cdot, ξ)] : R^{d} \to R^{d}$ is Lipschitz continuous with modulus $L > 0$ .

Assumption 4 (Pseudo-Monotonicity).

The averaged operator $T (\cdot) = E_{ξ} [F (\cdot, ξ)]$ is pseudo-monotone on $R^{d}$ , which means

\forall x, y \in R^{d} : ⟨ T (x), y - x ⟩ \geq 0 \Rightarrow ⟨ T (y), y - x ⟩ \geq 0.

At each iteration, the decision maker has access to a stochastic oracle, reporting an approximation of $T (x)$ of the form

{^T}_{n + 1} (x, ξ_{n + 1}) ≜ \frac{1}{m_{n + 1}} m_{n + 1} \sum i = 1 F (x, ξ_{n + 1}^{(i)}) for x \in R^{d} .

(3.1)

The sequence $(m_{n})_{n \geq 1} \subseteq N$ determines the batch size of the stochastic oracle. The random sequence $ξ_{n} = (ξ_{n}^{(1)}, \dots, ξ_{n}^{(m_{n})})$ is an i.i.d draw from $P$ . Approximations of the form (3.1

) are very common in Monte-Carlo simulation approaches, machine learning and computational statistics (see e.g.

[1, 2], and references therein); they are easy to obtain in case we are able to sample from the measure

P

. The forward-backward-forward algorithm requires two queries from the stochastic oracle in which mini-batch estimators of the averaged map

T

are revealed. This dynamic sampling strategy requires a sequence of integers

(m_{n})_{n \geq 1}

(the batch size) determining the size of the data set to be processed at each iteration. The random sample on each mini-batch consists of two independent stochastic processes

ξ_{n}

and

η_{n}

drawn from the law

P

, and explicitly given by

ξ_{n} ≜ (ξ_{n}^{(1)}, \dots, ξ_{n}^{(m_{n})}) % and η_{n} ≜ (η_{n}^{(1)}, \dots, η_{n}^{(m_{n})}) \forall n \geq 1.

Given the current position $X_{n}$ , Algorithm SFBF queries the SO once, to obtain the estimator $A_{n + 1} ≜ {^T}_{n + 1} (X_{n}, ξ_{n + 1})$ , and then constructs the random variable $Y_{n} = Π_{X} (X_{n} - α_{n} A_{n + 1})$ . Next, a second query to SO is made to obtain the estimator $B_{n + 1} ≜ {^T}_{n + 1} (Y_{n}, η_{n + 1})$ , followed by the update $X_{n + 1} = Y_{n} + α_{n} (A_{n + 1} - B_{n + 1})$ . The pseudocode for SFBF is given in Algorithm 1.

1:step-size sequence

α_{n}

; batch size sequence

m_{n}

2:Initialize

X

# initialization

3:for

n = 1, 2, \dots

4: Draw samples

ξ^{i}

and

η^{i}

from

P

(

i = 1, \dots, m_{n}

)

5: Oracle returns

A \leftarrow \frac{1}{m_{n}} m_{n} \sum i = 1 F (X, ξ^{i})

# first oracle query

6: Set

Y \leftarrow Π_{X} (X - α_{n} A)

# forward-backward step

7: Oracle returns

B \leftarrow \frac{1}{m_{n}} m_{n} \sum i = 1 F (Y, η^{i})

# second oracle query

8: Set

X \leftarrow Y + α_{n} (A - B)

# second forward step

9:end for

Algorithm 1 Stochastic forward-backward-forward (SFBF)

Observe that Algorithm SFBF is an infeasible method: the iterates $(X_{n})_{n \geq 0}$ are not necessarily elements of the admissible set $X$ , but the process $(Y_{n})_{n \geq 0}$ is by construction so. In the stochastic optimization case, i.e. for instances where $A_{n + 1}$

is an unbiased estimator of the gradient of a real-valued function, the process

(Y_{n})_{n \geq 0}

is seen to be a projected gradient step, where

A_{n + 1}

acts as an unbiased estimator for the stochastic gradient. This gradient step is used in an extrapolation step to generate the iterate

X_{n + 1}

. We just mention that related popular primal-dual splitting schemes like ADMM [3, 5] are infeasible by nature as well.

Assumption 5 (Step-size choice).

The step-size sequence $(α_{n})_{n \geq 0}$ in Algorithm SFBF satisfies

0 < α - - ≜ inf n \geq 0 α_{n} \leq ¯ α ≜ sup n \geq 1 α_{n} < \frac{1}{\sqrt{2} L} .

For $n \geq 0$ , we introduce the approximation error

W_{n + 1} ≜ A_{n + 1} - T (X_{n}), and Z_{n + 1} ≜ B_{n + 1} - T (Y_{n}),

(3.2)

and the sub-sigma algebras $(F_{n})_{n \geq 0}, ({^F}_{n})_{n \geq 0}$ , defined by $F_{0} ≜ σ (X_{0})$ , and

F_{n}

≜ σ (X_{0}, ξ_{1}, ξ_{2}, \dots, ξ_{n}, η_{1}, \dots, η_{n}) \forall n \geq 1,

and

{^F}_{n}

≜ σ (X_{0}, ξ_{1}, \dots, ξ_{n}, ξ_{n + 1}, η_{1}, \dots, η_{n}) \forall n \geq 0,

respectively. Observe that $F_{n} \subseteq {^F}_{n}$ for all $n \geq 0$ . We also define the filtrations $F ≜ (F_{n})_{n \geq 0}$ and $^F ≜ ({^F}_{n})_{n \geq 0}$ . The introduction of these two different sub-sigma algebras is important for many reasons. First, observe that they embody the information the learner has about the optimization problem. Indeed, the sub-sigma algebra $(F_{n})_{n \geq 0}$ corresponds to the information the decision maker has at the beginning the $n$ -th iteration, whereas $({^F}_{n})_{n \geq 0}$ is the information the decision maker has after the first (projection)-step of the iteration. Therefore, $(Y_{n})_{n \geq 0}$ is measurable with respect to the sub-sigma algebra $({^F}_{n})_{n \geq 0}$ and $(X_{n})_{n \geq 0}$ is measurable with respect to the sub-sigma algebra $(F_{n})_{n \geq 0}$ . Second, we see that the process $(W_{n})_{n \geq 1}$ is $F$ -adapted, whereas the process $(Z_{n})_{n \geq 1}$ is $^F$ -adapted, unbiased approximations relative to the respective information structures are provided:

E [W_{n + 1} | F_{n}] = 0 and E [Z_{n + 1} | {^F}_{n}] = 0 \forall n \geq 0.

Assumption 6 (Batch Size).

The batch size sequence $(m_{n})_{n \geq 1}$ satisfies $\sum_{n = 1}^{\infty} \frac{1}{m_{n}} < \infty$ .

A sufficient condition on the sequence $(m_{n})_{n \geq 1}$ is that for some constant $c > 0$ and integer $n_{0} > 0$ , we have

m_{n} = c \cdot (n + n_{0})^{1 + a} ln (n + n_{0})^{1 + b}

(3.3)

for $a > 0$ and $b \geq - 1$ , or $a = 0$ and $b > 0$ . The next assumption is essentially the same as the variance control assumption in [15].

Assumption 7 (Variance Control).

For all $x \in R^{d}$ and $p \geq 1$ , let

s_{p} (x) ≜ E [∥ F (x, ξ) - T (x) ∥^{p}]^{1 / p} .

There exist $p \geq 2, σ_{0} \geq 0$ , and a measurable locally bounded function $σ : X_{*} \to R_{+}$ such that for all $x \in R^{d}$ and all $x^{*} \in X_{*}$

s_{p} (x) \leq σ (x^{*}) + σ_{0} ∥ x - x^{*} ∥ .

(3.4)

Before we proceed with the convergence analysis, we want to make some clarifying remarks on this assumption. The most frequently used assumption on the SO’s approximation error, which dates back to the seminal work of Robbins and Monro (see [22, 9] for a textbook reference), asks for uniformly bounded variance (UBV), i.e.

sup x \in X s_{2} (x) \leq σ .

(UBV)

UBV is covered by creftype 7 when $σ_{0} = 0$ and ${sup}_{x \in X_{*}} σ (x^{*}) \leq σ$ . UBV is for instance valid when additive noise with finite $p$

-th moment is assumed, that is, for some random variable

ξ \in L^{2} (Ω, P; R^{d})

with

E [∥ ξ ∥^{p}]^{1 / p} \leq σ < \infty

we have

F (x, ξ) = T (x) + ξ P -a.s. .

However, assuming a global variance bound is not realistic in cases where the variance of the stochastic oracle depends on the position $x$ (see e.g. Example 1 in [17]). creftype 7 is much weaker than UBV, as it exploits the local variance of the stochastic oracle rather than, potentially hard to estimate, global mean square variance bounds. The recent papers [15, 17] make similar assumptions on the variance of the stochastic oracle. It is shown there that creftype 7 is most natural in cases where the feasible set $X$ is unbounded, and it is always satisfied when the Carathéodory functions $F (\cdot, ξ)$ are random Lipschitz (see Example 3.1 below). Since Algorithm 1 is an infeasible method, we are forced to analyze the behavior of the stochastic process $(X_{n}, Y_{n})_{n \geq 0}$ on an unbounded domain, which makes creftype 7 the only realistic and convenient choice for us. Example 3.1 illustrates an important instance where creftype 7 holds.

Example 3.1.

Assume for the Carathéodory mapping $F : R^{d} \times Ξ \to R^{d}$ that there exists $L \in L^{p} (Ω, F, P; R_{+})$ with

∥ F (x, ξ) - F (y, ξ) ∥ \leq L (ξ) ∥ x - y ∥ \forall x, y \in R^{d} .

Call $L$ the Lipschitz constant of the map $x \mapsto T (x) = E [F (x, ξ)]$ . Then, a repeated application of the Minkowski inequality shows that for all $x \in R^{d}$ and all $x^{*} \in X_{*}$ we have

	$s_{p} (x)$	$\leq E [∥ F (x, ξ) - F (x^{}, ξ) ∥^{p}]^{1 / p} + s_{p} (x^{}) + ∥ T (x) - T (x^{*}) ∥$
		$\leq (E [L (ξ)^{p}]^{1 / p} + L) ∥ x - x^{} ∥ + s_{p} (x^{}) .$

Let $σ (x^{*})$ denote a bound on $s_{p} (x^{*})$ and set $σ_{0} ≜ L + E [L (ξ)^{p}]^{1 / p}$ , to get a variance bound as required in creftype 7.

4. Convergence Analysis

We consider the quadratic residual function defined by

r_{a} (x)^{2} ≜ ∥ x - Π_{X} (x - a T (x)) ∥^{2} \forall x \in R^{d} .

The reader familiar with the literature on finite-dimensional variational inequalities will recognize this immediately as the energy defined by the natural map $F_{a}^{nat} (x) ≜ x - Π_{X} (x - a T (x))$ [11, chapter 10]. It is well known that $r_{a} (x)$ is a merit function for $VI (T, X)$ . Moreover, ${r_{a} (x); a > 0}$ is a family of equivalent merit functions for $VI (T, X)$ , in the sense that $r_{b} (x) \geq r_{a} (x)$ for all $b > a > 0$ [11, Proposition 10.3.6]. Denote

ρ_{n} ≜ 1 - 2 L^{2} α_{n}^{2} \forall n \geq 0.

(4.1)

We define recursively the process $(V_{n})_{n \geq 0}$ by $V_{0} = 0$ and, for all $n \geq 1$ ,

V_{n + 1} ≜ V_{n} + (4 + ρ_{n}) α_{n}^{2} ∥ W_{n + 1} ∥^{2} + 4 α_{n}^{2} ∥ Z_{n + 1} ∥^{2},

so that

Δ V_{n} ≜ V_{n + 1} - V_{n} = (4 + ρ_{n}) α_{n}^{2} ∥ W_{n + 1} ∥^{2} + 4 α_{n}^{2} ∥ Z_{n + 1} ∥^{2} \forall n \geq 0.

(4.2)

Additionally, we define for all $x \in R^{d}$ the process $(U_{n} (x))_{n \geq 0}$ given by $U_{0} (x) = 0$ , and

U_{n + 1} (x) ≜ U_{n} (x) + 2 α_{n} ⟨ Z_{n + 1}, x - Y_{n} ⟩ \forall n \geq 1

with corresponding increment

Δ U_{n} (x) ≜ 2 α_{n} ⟨ Z_{n + 1}, x - Y_{n} ⟩ \forall n \geq 0.

For any reference point $x \in R^{d}$ we see that $E [Δ U_{n} (x) | {^F}_{n}] = 0$ for all $n \geq 0$ . Hence, the process ${(U_{n} (x))}_{n \geq 0}$ is a martingale w.r.t. the filtration $^F$ . Since $F_{n} \subseteq {^F}_{n}$ , the tower property implies that

E [Δ U_{n} (x) | F_{n}] = 0 \forall x \in R^{d} \forall n \geq 0,

(4.3)

showing that it is also a $F$ -martingale. $(V_{n})_{n \geq 0}$ is an increasing process, with increments $Δ V_{n}$ whose expected value is determined by the variance of the approximation error of the stochastic oracle feedback. In terms of these increment processes, we establish the following fundamental recursion.

Lemma 4.1.

For all $x^{*} \in X_{*}$ and all $n \geq 0$ we have

∥ X_{n + 1} - x^{*} ∥^{2} \leq ∥ X_{n} - x^{*} ∥^{2} - \frac{ρ_{n}}{2} r_{α_{n}} (X_{n})^{2} + Δ U_{n} (x^{*}) + Δ V_{n} P - a.s. .

(4.4)

Proof.

This recursive relation follows via several simple algebraic steps. Let be $x^{*} \in X_{*}$ and $n \geq 0$ fixed.

Step 1

We have

⟨ T (x^{*}), y - x^{*} ⟩ \geq 0 \forall y \in X .

Using that $α_{n} > 0$ as well as the pseudo-monotonicity of $T$ , we see

⟨ α_{n} T (Y_{n}), Y_{n} - x^{*} ⟩ \geq 0.

Using the Doob decomposition in equation (3.2), we can rewrite this inequality as

⟨ α_{n} B_{n + 1}, Y_{n} - x^{*} ⟩ \geq α_{n} ⟨ Z_{n + 1}, Y_{n} - x^{*} ⟩ .

(4.5)

Since $Y_{n} = Π_{X} (X_{n} - α_{n} A_{n + 1})$ , from Lemma 2.1(i) we conclude that

⟨ x^{*} - Y_{n}, Y_{n} - X_{n} + α_{n} A_{n + 1} ⟩ \geq 0.

(4.6)

Adding (4.5) and (4.6) gives

⟨ α_{n} (A_{n + 1} - B_{n + 1}) - X_{n} + Y_{n}, x^{*} - Y_{n} ⟩ \geq α_{n} ⟨ Z_{n + 1}, Y_{n} - x^{*} ⟩,

which is equivalent to

⟨ x^{*} - Y_{n}, X_{n + 1} - X_{n} ⟩ \geq α_{n} ⟨ Z_{n + 1}, Y_{n} - x^{*} ⟩ .

(4.7)

Step 2

Using (4.7), we get

	$⟨ X_{n + 1} - X_{n}, X_{n + 1} - x^{*} ⟩ =$	$⟨ X_{n + 1} - X_{n}, Y_{n} - x^{*} ⟩ + ⟨ X_{n + 1} - X_{n}, X_{n + 1} - Y_{n} ⟩$
	$\leq$	$⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩ + ∥ X_{n + 1} - X_{n} ∥^{2}$
		$+ ⟨ X_{n + 1} - X_{n}, X_{n} - Y_{n} ⟩$
	$=$	$⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩ + ∥ X_{n + 1} - X_{n} ∥^{2} - ∥ X_{n} - Y_{n} ∥^{2}$
		$+ α_{n} ⟨ A_{n + 1} - B_{n + 1}, X_{n} - Y_{n} ⟩$

where we have used the definition of $X_{n + 1}$ in the last equality. The Pythagoras identity in Lemma 2.2 gives us

	$∥ X_{n + 1} - x^{*} ∥^{2} =$	$∥ X_{n} - x^{} ∥^{2} - ∥ X_{n + 1} - X_{n} ∥^{2} + 2 ⟨ X_{n + 1} - X_{n}, X_{n + 1} - x^{} ⟩$
	$\leq$	$∥ X_{n} - x^{*} ∥^{2} + ∥ X_{n + 1} - X_{n} ∥^{2} - 2 ∥ X_{n} - Y_{n} ∥^{2}$
		$+ 2 ⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩ + 2 α_{n} ⟨ A_{n + 1} - B_{n + 1}, X_{n} - Y_{n} ⟩ .$

Step 3.

Using again the definition of $X_{n + 1}$ , we see

	$∥ X_{n + 1} - X_{n} ∥^{2} =$	$∥ Y_{n} + α_{n} (A_{n + 1} - B_{n + 1}) - X_{n} ∥^{2}$
	$=$	$∥ X_{n} - Y_{n} ∥^{2} + α_{n}^{2} ∥ A_{n + 1} - B_{n + 1} ∥^{2} + 2 α_{n} ⟨ A_{n + 1} - B_{n + 1}, Y_{n} - X_{n} ⟩$
	$\leq$	$∥ X_{n} - Y_{n} ∥^{2} + 2 α_{n}^{2} ∥ T (X_{n}) - T (Y_{n}) ∥^{2} + 2 α_{n}^{2} ∥ W_{n + 1} - Z_{n + 1} ∥^{2}$
		$+ 2 α_{n} ⟨ A_{n + 1} - B_{n + 1}, Y_{n} - X_{n} ⟩$
	$\leq$	$∥ X_{n} - Y_{n} ∥^{2} + 2 L^{2} α_{n}^{2} ∥ X_{n} - Y_{n} ∥^{2} + 4 α_{n}^{2} ∥ W_{n + 1} ∥^{2} + 4 α_{n}^{2} ∥ Z_{n + 1} ∥^{2}$
		$+ 2 α_{n} ⟨ A_{n + 1} - B_{n + 1}, Y_{n} - X_{n} ⟩ .$

The first inequality is the Cauchy-Schwarz inequality. The second inequality follows from the $L$ -Lipschitz continuity of the averaged operator $T$ (creftype 3), and again the Cauchy-Schwarz inequality. Combining this with the last inequality obtained in Step 2, we see that

	$∥ X_{n + 1} - x^{*} ∥^{2}$	$\leq ∥ X_{n} - x^{*} ∥^{2} - (1 - 2 L^{2} α_{n}^{2}) ∥ X_{n} - Y_{n} ∥^{2} + 4 α_{n}^{2} ∥ W_{n + 1} ∥^{2}$
		$+ 4 α_{n}^{2} ∥ Z_{n + 1} ∥^{2} + 2 ⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩ .$

Step 4

By the definition of the squared residual function, the definition of $Y_{n}$ and Lemma 2.1(iii), we have

	$r_{α_{n}} (X_{n})^{2}$	$= ∥ X_{n} - Π_{X} (X_{n} - α_{n} T (X_{n})) ∥^{2}$
		$\leq 2 ∥ X_{n} - Y_{n} ∥^{2} + 2 ∥ Y_{n} - Π_{X} (X_{n} - α_{n} T (X_{n})) ∥^{2}$
		$= 2 ∥ X_{n} - Y_{n} ∥^{2} + 2 ∥ Π_{X} (X_{n} - α_{n} A_{n + 1}) - Π_{X} (X_{n} - α_{n} T (X_{n}) ∥^{2}$
		$\leq 2 ∥ X_{n} - Y_{n} ∥^{2} + 2 ∥ α_{n} W_{n + 1} ∥^{2} .$

Hence,

- 2 ∥ X_{n} - Y_{n} ∥^{2} \leq 2 α_{n}^{2} ∥ W_{n + 1} ∥^{2} - r_{α_{n}} (X_{n})^{2} .

(4.8)

Step 5

Combining (4.8) with the last inequality from Step 3 and recalling creftype 5, we conclude

	$∥ X_{n + 1} - x^{*} ∥^{2} \leq$	$∥ X_{n} - x^{*} ∥^{2} - \frac{1}{2} (1 - 2 L^{2} α_{n}^{2}) r_{α_{n}} (X_{n})^{2} + (1 - 2 L^{2} α_{n}^{2}) α_{n}^{2} ∥ W_{n + 1} ∥^{2}$
		$+ 4 α_{n}^{2} ∥ W_{n + 1} ∥^{2} + 4 α_{n} ∥ Z_{n + 1} ∥^{2} + 2 ⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩$
	$=$	$∥ X_{n} - x^{*} ∥^{2} - \frac{ρ_{n}}{2} r_{α_{n}} (X_{n})^{2} + (4 + ρ_{n}) (α_{n})^{2} ∥ W_{n + 1} ∥^{2} + 4 α_{n}^{2} ∥ Z_{n + 1} ∥^{2}$
		$+ 2 ⟨ α_{n} Z_{n + 1}, x^{*} - Y_{n} ⟩ .$

The definitions of the increments associated with the martingales $(U_{n} (x^{*}))_{n \geq 0}$ and $(V_{n})_{n \geq 0}$ give the claimed result. $■$

Remark 4.1.

One can notice that in the above proof the pseudo-monotonicity of $T$ is used only in Step 1 of the above proof, if order to obtain relation (4.5). Thus, as happened in [8, 32], the pseudo-monotonicity of $T$ can actually be replaced by the following weaker assumption

⟨ T (x), x - x^{*} ⟩ \geq 0 \forall x \in X, x^{*} \in X_{*} .

Forward-backward-forward methods with variance reduction for stochastic variational inequalities

Stochastic Variance Reduction for Variational Inequality Methods

A Relaxed Inertial Forward-Backward-Forward Algorithm for Solving Monotone Inclusions with Application to GANs

Simple and optimal methods for stochastic variational inequalities, I: operator extrapolation

On the convergence of single-call stochastic extra-gradient methods

A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operators

Stochastic Conditional Gradient++

Adaptive and Universal Single-gradient Algorithms for Variational Inequalities

1. Introduction

Definition 1.1.

1.1. Our Contribution

2. Preliminaries

2.1. Notation

Lemma 2.1.

Remark 2.1.

Lemma 2.2 (Pythagorean identity).

2.2. Probabilistic Tools

Lemma 2.3 (Robbins-Siegmund).

Lemma 2.4.

3. The stochastic forward-backward-forward algorithm

Assumption 1 (Consistency).

Assumption 2 (Stochastic Model).

Assumption 3 (Lipschitz continuity).

Assumption 4 (Pseudo-Monotonicity).

Assumption 5 (Step-size choice).

Assumption 6 (Batch Size).

Assumption 7 (Variance Control).

Example 3.1.

4. Convergence Analysis

Lemma 4.1.

Proof.

Step 1

Step 2

Step 3.

Step 4

Step 5

Remark 4.1.

Forward-backward-forward methods with variance reduction for stochastic variational inequalities

Related Research

Stochastic Variance Reduction for Variational Inequality Methods

A Relaxed Inertial Forward-Backward-Forward Algorithm for Solving Monotone Inclusions with Application to GANs

Simple and optimal methods for stochastic variational inequalities, I: operator extrapolation

On the convergence of single-call stochastic extra-gradient methods

A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operators

Stochastic Conditional Gradient++

Adaptive and Universal Single-gradient Algorithms for Variational Inequalities

1. Introduction

Definition 1.1.

1.1. Our Contribution

2. Preliminaries

2.1. Notation

Lemma 2.1.

Remark 2.1.

Lemma 2.2 (Pythagorean identity).

2.2. Probabilistic Tools

Lemma 2.3 (Robbins-Siegmund).

Lemma 2.4.

3. The stochastic forward-backward-forward algorithm

Assumption 1 (Consistency).

Assumption 2 (Stochastic Model).

Assumption 3 (Lipschitz continuity).

Assumption 4 (Pseudo-Monotonicity).

Assumption 5 (Step-size choice).

Assumption 6 (Batch Size).

Assumption 7 (Variance Control).

Example 3.1.

4. Convergence Analysis

Lemma 4.1.

Proof.

Step 1

Step 2

Step 3.

Step 4

Step 5

Remark 4.1.