Deterministic Approximation of Random Walks in Small Space

03/15/2019 ∙ by Jack Murtagh, et al. ∙ Harvard University Stanford University 0

We give a deterministic, nearly logarithmic-space algorithm that given an undirected graph G, a positive integer r, and a set S of vertices, approximates the conductance of S in the r-step random walk on G to within a factor of 1+ϵ, where ϵ>0 is an arbitrarily small constant. More generally, our algorithm computes an ϵ-spectral approximation to the normalized Laplacian of the r-step walk. Our algorithm combines the derandomized square graph operation (Rozenman and Vadhan, 2005), which we recently used for solving Laplacian systems in nearly logarithmic space (Murtagh, Reingold, Sidford, and Vadhan, 2017), with ideas from (Cheng, Cheng, Liu, Peng, and Teng, 2015), which gave an algorithm that is time-efficient (while ours is space-efficient) and randomized (while ours is deterministic) for the case of even r (while ours works for all r). Along the way, we provide some new results that generalize technical machinery and yield improvements over previous work. First, we obtain a nearly linear-time randomized algorithm for computing a spectral approximation to the normalized Laplacian for odd r. Second, we define and analyze a generalization of the derandomized square for irregular graphs and for sparsifying the product of two distinct graphs. As part of this generalization, we also give a strongly explicit construction of expander graphs of every size.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Random walks provide the most dramatic example of the power of randomized algorithms for solving computational problems in the space-bounded setting, as they only require logarithmic space (to store the current state or vertex). In particular, since undirected graphs have polynomial cover time, random walks give a randomized logspace () algorithm for Undirected S-T Connectivity [AKL]. Reingold [Rei] showed that this algorithm can be derandomized, and hence that Undirected S-T Connectivity is in deterministic logspace (). However, Reingold’s algorithm does not match the full power of random walks on undirected graphs; in particular it does not allow us to approximate properties of the random walk at lengths below the mixing time.

In this work, we provide a nearly logarithmic-space algorithm for approximating properties of arbitrary-length random walks on an undirected graph, in particular the conductance of any set of vertices:

Definition 1.1.

Let be an undirected graph, a positive integer, and a set of vertices. The conductance of under the -step random walk on is defined as

where is a random walk on started at the stationary distribution .

Theorem 1.2.

There is a deterministic algorithm that given an undirected multigraph on vertices, a positive integer , a set of vertices , and , computes a number such that

and runs in space , where is the bit length of the input graph .

Previously, approximating conductance could be done in space, which follows from Saks’ and Zhou’s proof that is in [SZ].

Two interesting parameter regimes where we improve the Saks-Zhou bound are when , in which case our algorithm runs in space , or when and , in which case our algorithm runs in space . When exceeds the time for random walks on undirected graphs to mix to within distance of the stationary distribution, the conductance can be approximated in space by using Reingold’s algorithm to find the connected components of

, and the bipartitions of the components that are bipartite and calculating the stationary probability of

restricted to each of these pieces, which is proportional to the sum of degrees of vertices in .

We prove Theorem 1.2 by providing a stronger result that with the same amount of space it is possible to compute an -spectral approximation to the normalized Laplacian of the -step random walk on .

Definition 1.3.

Let be an undirected graph with adjacency matrix , diagonal degree matrix , and transition matrix . The transition matrix for the -step random walk on is . The normalized Laplacian of the -step random walk is the symmetric matrix for .

Note that the normalized Laplacian can also be expressed as , so it does indeed capture the behavior of -step random walks on .111When is irregular, the matrix is not necessarily symmetric. It is a directed Laplacian as defined in [CKP2, CKP1]. See Definition 2.5.

Theorem 1.4 (Main result).

There is a deterministic algorithm that given an undirected multigraph on vertices with normalized Laplacian , a nonnegative integer , and , constructs an undirected multigraph whose normalized Laplacian is an -spectral approximation of

. That is, for all vectors

The algorithm runs in space , where is the bit length of the input graph .

Theorem 1.2 follows from Theorem 1.4 by taking to be where is the characteristic vector of the set and normalizing appropriately (See Section 5).

Our main technique for proving Theorem 1.4 is the derandomized product, a new generalization of the derandomized square, which was introduced by Rozenman and Vadhan [RV] to give an alternative proof that
Undirected S-T Connectivity is in . Our main result follows from carefully applying the derandomized product and analyzing its properties with inequalities from the theory of spectral approximation. Specifically, our analysis is inspired by the work of Cheng, Cheng, Liu, Peng, and Teng [CCL2], who studied the approximation of random walks by randomized algorithms running in nearly linear time. We emphasize that the work of [CCL2] gives a randomized algorithm with high space complexity (but low time complexity) for approximating properties of even length walks while we give a deterministic, space-efficient algorithm for approximating properties of walks of every length. Interestingly, while the graphs in our results are all undirected, some of our analyses use techniques for spectral approximation of directed graphs introduced by Cohen, Kelner, Peebles, Peng, Rao, Sidford, and Vladu [CKP2, CKP1].

The derandomized square can be viewed as applying the pseudorandom generator of Impagliazzo, Nisan, and Wigderson [INW] to random walks on labelled graphs. It is somewhat surprising that repeated derandomized squaring does not blow up the error by a factor proportional to the length of the walk being derandomized. For arbitrary branching programs, the INW generator does incur error that is linear in the length of the program. Some special cases such as regular [BRRY, BV, De] and permutation [De, Ste] branching programs of constant width have been shown to have a milder error growth as a function of the walk length. Our work adds to this list by showing that properties of random walks of length

on undirected graphs can be estimated in terms of spectral approximation without error accumulating linearly in

.

In our previous work [MRSV], we showed that the Laplacian of the derandomized square of a regular graph spectrally approximates the Laplacian of the true square, , and this was used in a recursion from [PS] to give a nearly logarithmic-space algorithm for approximately solving Laplacian systems . A natural idea to approximate the Laplacian of higher powers, , is to repeatedly derandomized square. This raises three challenges, and we achieve our result by showing how to overcome each:

  1. It is not guaranteed from [MRSV] that repeated derandomized squaring preserves spectral approximation. For this, we use ideas from [CCL2] to argue that it does.

  2. When is not a power of 2, the standard approach would be to write where is the th bit of and multiply approximations to for all such that . The problem is that multiplying spectral approximations of matrices does not necessarily yield a spectral approximation of their product. Our solution is to generalize the derandomized square to produce sparse approximations to the product of distinct graphs. In particular, given and an approximation to , our derandomized product allows us to combine and to approximate . Although our generalized graph product is defined for undirected graphs, its analysis uses machinery for spectral approximation of directed graphs, introduced in [CKP1].

  3. We cannot assume that our graph is regular without loss of generality. In contrast, [Rei, RV, MRSV] could do so, since adding self-loops does not affect connectivity or solutions to Laplacian systems of , however, it does affect random walks. Our solution is to define and analyze the derandomized product for irregular graphs.

A key element in the derandomized product is a strongly explicit (i.e. neighbor relations can be computed in space ) construction of expander graphs whose sizes equal the degrees of the vertices in the graphs being multiplied. This is problematic when we are not free to add self loops to the graphs because strongly explicit constructions of expander graphs only exist for graph sizes that are certain subsets of such as powers of 2 (Cayley graphs based on [NN] and [AGHP]), perfect squares [Mar, GG], and other size distributions [RVW] or are only explicit in the sense of running time or parallel work [LPS]. To address this issue, we give a strongly explicit construction of expander graphs of all sizes by giving a reduction from existing strongly explicit constructions in Section 3.

Many of our techniques are inspired by Cheng, Cheng, Liu, Peng, and Teng [CCL2], who gave two algorithms for approximating random walks. One is a nearly linear time randomized algorithm for approximating random walks of even length and another works for all walk lengths but has a running time that is quadratic in , and so only yields a nearly linear time algorithm for that is polylogarithmic in the size of the graph. In addition, [JKPS] studied the problem of computing sparse spectral approximations of random walks but the running time in their work also has a quadratic dependence on . We extend these results by giving a nearly linear time randomized algorithm for computing a spectral approximation to for all . This is discussed in Section 5.

2 Preliminaries

2.1 Spectral Graph Theory

Given an undirected multigraph the Laplacian of is the symmetric matrix , where is the diagonal matrix of vertex degrees and is the adjacency matrix of . The transition matrix of the random walk on is . is the probability that a uniformly random edge from vertex leads to vertex (i.e. the number of edges between and divided by the degree of ). The normalized Laplacian of is the symmetric matrix . Note that when is regular, the matrix . The transition matrix of the -step random walk on is

. For all probability distributions

, is the distribution over vertices that results from picking a random vertex according to and then running a random walk on for steps. The transition matrix of the -step random walk on is related to the normalized Laplacian in the following way:

For undirected multigraphs, the matrix

has real eigenvalues between

and and so has eigenvalues in and thus is positive semidefinite (PSD). The spectral norm of a real matrix , denoted

, is the largest singular value of

. That is, the square root of the largest eigenvalue of . When is symmetric, equals the largest eigenvalue of in absolute value. For an undirected graph with adjacency matrix , we write to denote the graph with adjacency matrix , i.e. the multigraph with all edges duplicated to have multiplicity .

Given a symmetric matrix , its Moore-Penrose Pseudoinverse, denoted

, is the unique matrix with the same eigenvectors as

such that for each eigenvalue of , the corresponding eigenvalue of is if and 0 otherwise. When is a Laplacian, we write to denote the unique symmetric PSD matrix square root of the pseudoinverse of .

To measure the approximation between graphs we use spectral approximation222In [MRSV], we use an alternative definition of spectral approximation where if for all , . We find Definition 2.1 more convenient for this paper.[ST]:

Definition 2.1.

Let be symmetric PSD matrices. We say that is an -approximation of (written ) if for all vectors

Note that Definition 2.1 is not symmetric in and . Spectral approximation can also be written in terms of the Loewner partial ordering of PSD matrices:

where for two matrices , we write if is PSD. Spectral approximation has a number of useful properties listed in the following proposition.

Proposition 2.2.

If are PSD symmetric matrices then:

  1. If for then

  2. If and then ,

  3. If and is any matrix then ,

  4. If then ,

  5. If and then , and

  6. If then for all nonnegative scalars

For regular undirected graphs, we use the measure introduced by [Mih]

for the rate at which a random walk converges to the uniform distribution.

Definition 2.3 ([Mih]).

Let be a regular undirected graph with transition matrix . Define

is called the spectral gap of .

is known to be a measure of how well-connected a graph is. The smaller , the faster a random walk on converges to the uniform distribution. Graphs with bounded away from 1 are called expanders. Expanders can equivalently be characterized as graphs that spectrally approximate the complete graph. This is formalized in the next lemma.

Lemma 2.4.

Let be a -regular undirected multigraph on vertices with transition matrix and let be a matrix with in every entry (i.e. is the transition matrix of the complete graph with a self loop on every vertex). Then if and only if .

A proof of Lemma 2.4 can be found in Appendix A. In [CKP1] Cohen, Kelner, Peebles, Peng, Rao, Sidford, and Vladu introduced a definition of spectral approximation for asymmetric matrices. Although the results in our paper only concern undirected graphs, some of our proofs use machinery from the theory of directed spectral approximation.

Definition 2.5 (Directed Laplacian [Ckp2, Ckp1]).

A matrix is called a directed Laplacian if for all and for all . The associated directed graph has vertices and an edge of weight for all with .

Definition 2.6 (Asymmetric Matrix Approximation [Ckp1]).

Let and be (possibly asymmetric) matrices such that is PSD. We say that is a directed -approximation of if:

  1. , and

Below we state some useful lemmas about directed spectral approximation. The first gives an equivalent formulation of Definition 2.6.

Lemma 2.7 ([Ckp1] Lemma 3.5).

Let be a (possibly asymmetric) matrix and let . A matrix is a directed -approximation of if and only if for all vectors

Lemma 2.8 ([Ckp1] Lemma 3.6).

Suppose is a directed -approximation of and let and . Then .

Lemma 2.8 says that directed spectral approximation implies the usual notion from Definition 2.1 for “symmetrized” versions of the matrices and . In fact, when the matrices and are both symmetric, the two definitions are equivalent:

Lemma 2.9.

Let and be symmetric PSD matrices. Then is a directed -approximation of if and only if .

A proof of Lemma 2.9 can be found in Appendix B.

2.2 Space Bounded Computation

We use a standard model of space-bounded computation where the machine has a read-only input tape, a constant number of read/write work tapes, and a write-only output tape. If throughout every computation on inputs of length at most , uses at most total tape cells on all the work tapes, we say runs in space . Note that may write more than cells (in fact as many as ) but the output tape is write-only. The following proposition describes the behavior of space complexity when space bounded algorithms are composed.

Proposition 2.10.

Let , be functions that can be computed in space , respectively, and has output of length at most on inputs of length . Then can be computed in space

2.3 Rotation Maps

In the space-bounded setting, it is convenient to use local descriptions of graphs. Such descriptions allow us to navigate large graphs without loading them entirely into memory. For this we use rotation maps, functions that describe graphs through their neighbor relations. Rotation maps are defined for graphs with labeled edges as described in the following definition.

Definition 2.11 ([Rvw]).

A two-way labeling of an undirected multigraph with vertex degrees , is a labeling of the edges in such that

  1. Every edge has two labels: one in as an edge incident to and one in as an edge incident to ,

  2. For every vertex , the labels of the edges incident to are distinct.

In [RV], two-way labelings are referred to as undirected two-way labelings. Note that every graph has a two-way labeling where each vertex “names” its neighbors uniquely in some canonical way based on the order they’re represented in the input. We will describe multigraphs with two-way labelings using rotation maps:

Definition 2.12 ([Rvw]).

Let be an undirected multigraph on vertices with a two-way labeling. The rotation map Rot is defined as follows: Rot if the th edge to vertex leads to vertex and this edge is the th edge incident to .

We will use expanders that have efficiently computable rotation maps. We call such graphs strongly explicit. The usual definition of strong explicitness only refers to time complexity, but we will use it for both time and space.

Definition 2.13.

A family of two-way labeled graphs , where is a -regular graph on vertices, is called strongly explicit if given , a vertex and an edge label , Rot can be computed in time and space .

3 The Derandomized Product and Expanders of All Sizes

In this section we introduce our derandomized graph product. The derandomized product generalizes the derandomized square graph operation that was introduced by Rozenman and Vadhan [RV] to give an alternative proof that Undirected S-T Connectivity is in . Unlike the derandomized square, the derandomized product is defined for irregular graphs and produces a sparse approximation to the product of any two (potentially different) graphs with the same vertex degrees.

Here, by the ‘product’ of two graphs

, we mean the reversible Markov chain with transitions defined as follows: from a vertex

, with probability take a random step on followed by a random step on and with probability take a random step on followed by a random step on .

When , this is the same as taking a 2-step random walk on . Note, however, that when is irregular, a 2-step random walk is not equivalent to doing a 1-step random walk on the graph , whose edges correspond to paths of length 2 in . Indeed, even the stationary distribution of the random walk on may be different than on .333For example, let be the graph on two vertices with one edge connecting them and a single self loop on . Then is the stationary distribution of and is the stationary distribution of . Nevertheless, our goal in the derandomized product is to produce a relatively sparse graph whose 1-step random walk approximates the 2-step random walk on .

The intuition behind the derandomized product is as follows: rather than build a graph with every such two-step walk, we use expander graphs to pick a pseudorandom subset of the walks. Specifically, we first pick at random. Then, as before we take a truly random step from to in . But for the second step, we don’t use an arbitrary edge leaving in , but rather correlate it to the edge on which we arrived at using a -regular expander on deg vertices, where we assume that the vertex degrees in and are the same. When , the vertex degrees of the resulting two-step graph will be sparser than without derandomization. However using the pseudorandom properties of expander graphs, we can argue that the derandomized product is a good approximation of the true product.

Definition 3.1 (Derandomized Product).

Let be undirected multigraphs on vertices with two-way labelings and identical vertex degrees . Let be a family of two-way labeled, -regular expanders of sizes including . The derandomized product with respect to , denoted , is an undirected multigraph on vertices with vertex degrees and rotation map Rot defined as follows: For , , and we compute Rot as

  1. Let Rot

  2. Let Rot

  3. Let Rot

  4. Output

where denotes the bit-negation of .

Note that when the derandomized product generalizes the derandomized square [RV] to irregular graphs, albeit with each edge duplicated twice. Note that Definition 3.1 requires that each vertex has the same degree in and , ensuring that the random walks on , and all have the same stationary distribution. This can be generalized to the case that there is an integer such that for each vertex with degree in , has degree in . For this, we can duplicate each edge in times to match the degrees of and then apply the derandomized product to the result. In such cases we abuse notation and write to mean .

In [MRSV] we showed that the derandomized square produces a spectral approximation to the true square. We now show that the derandomized product also spectrally approximates a natural graph product.

Theorem 3.2.

Let be undirected multigraphs on vertices with two-way labelings, and normalized Laplacians and . Let have vertex degrees and have vertex degrees where for all , for a positive integer . Let be a family of two-way labeled, -regular expanders with for all , of sizes including . Let be the normalized Laplacian of . Then

Proof of Theorem 3.2.

Note that has the same transition matrix and normalized Laplacian as . So we can replace with and assume without loss of generality.

Since and have the same vertex degrees, we can we write

(1)

where and are the transition matrices of and , respectively.

Following the proofs in [RV] and [MRSV], we can write the transition matrix for the random walk on as , where each matrix corresponds to a step in the definition of the derandomized product. The two terms correspond to and in the derandomized product and, setting ,

  • is a matrix that “lifts” a probability distribution over to one over where the mass on each coordinate is divided uniformly over the corresponding degree . That is, if and 0 otherwise where the rows of are ordered .

  • and are the symmetric permutation matrices corresponding to the rotation maps of and , respectively. That is, entry in is if Rot and 0 otherwise for .

  • is a symmetric block-diagonal matrix with blocks where block is the transition matrix for the random walk on , the expander in our family with vertices.

  • is the matrix that maps any -vector to an -vector by summing all the entries corresponding to edges incident to the same vertex in and . This corresponds to projecting a distribution on back down to a distribution over . if and 0 otherwise where the columns of are ordered .

Likewise, we can write

(2)

where is a symmetric block-diagonal matrix with blocks where block is , the transition matrix for the complete graph on vertices with a self loop on every vertex. That is, every entry of is .

We will show that

From this the theorem follows by multiplying by on the left and on the right and applying Proposition 2.2 Part 3. Since , the left-hand side becomes

where is the diagonal matrix of vertex degrees of . By Equations (1) and (2), the right-hand side becomes .

By Lemma 2.4, each graph in is a -approximation of the complete graph on the same number of vertices. It follows that because the quadratic form of a block diagonal matrix equals the sum of the quadratic forms of its blocks. By Lemma 2.9 and the fact that is PSD, is also a directed -approximation of . So for all vectors we have

The first inequality uses Lemma 2.7. We can add the absolute values on the left-hand side since the right-hand side is always nonnegative ( is PSD) and invariant to swapping with . The second inequality follows from the fact that is PSD and so

Fix and set and . Recall that and are symmetric permutation matrices and hence . Also note that for all square matrices and vectors , . Combining these observations with the above gives

Rearranging the above shows that

which proves the theorem. ∎

Note that for a graph with normalized Laplacian and transition matrix , approximating as in Theorem 3.2 for and gives a form of approximation to random walks of length on , as

To apply the derandomized product, we need an expander family with sizes equal to all of the vertex degrees. However, existing constructions of strongly explicit expander families only give graphs of sizes that are subsets of such as all powers of 2 or all perfect squares. In [RV, MRSV] this was handled by adding self loops to make the vertex degrees all equal and matching the sizes of expanders in explicit families. Adding self loops was acceptable in those works because it does not affect connectivity (the focus of [RV]) or the Laplacian (the focus of [MRSV]). However it does affect long random walks (our focus), so we cannot add self loops. Instead, we show how to obtain strongly explicit expanders of all sizes. Our construction works by starting with a strongly explicit expander from one of the existing constructions and merging vertices to achieve any desired size:

Theorem 3.3.

There exists a family of strongly explicit expanders such that for all and there is a and a -regular graph on vertices with .

Proof.

Let be a -regular expander on vertices such that , is a constant independent of and . can be constructed using already known strongly explicit constructions such as [GG, RVW] followed by squaring the graph a constant number of times to achieve . We will construct as follows: Pair off the first vertices with the last vertices in and merge each pair into a single vertex (which will then have degree ). To make the graph regular, add self loops to all of the unpaired vertices. More precisely, given and we compute Rot as follows:

  1. If [ is a paired vertex]:

    1. If , let , [ is the first vertex in pair]

    2. else let , [ is the second vertex in pair]

    3. let

  2. else (if ) [ is an unpaired vertex]

    1. If , let , , and [original edge]

    2. else let [new self loop]

    1. If , let

    2. else let , .

  3. Output

Next we show that is bounded below 1 by a constant. The theorem then follows by taking the th power to drive below . This gives the graph degree .

Let be the adjacency matrix of and be the all ones matrix. Since , Lemma 2.4 implies that

Define to be the matrix such that if and only if vertex was merged into vertex or vertex was not merged and is labeled vertex in . That is, if and only if or . Then the unnormalized Laplacian of the expander after the merging step is . Adding self loops to a graph does not change its Laplacian. So applying Proposition 2.2 parts 3 and 6 we get

Note that the righthand side is the normalized Laplacian of the graph that results from starting with the complete graph on vertices, merging the same pairs of vertices that are merged in and adding self loops to all of the unmerged vertices for regularity.

We finish the proof by showing that and thus is a -approximation of the complete graph by Proposition 2.2 Part 2 and Lemma 2.4. Recalling that completes the proof.

has at least edges between every pair of vertices so we can write its transition matrix as

where is the transition matrix of the complete graph on vertices with self loops on every vertex and is the transition matrix for an -regular multigraph. Since the uniform distribution is stationary for all regular graphs, is an eigenvector of eigenvalue 1 for and . Thus

which completes the proof. ∎

4 Main Result

In this section we prove Theorem 1.4, our main result regarding space bounded computation of the normalized Laplacian of the -step random walk on .

Theorem 1.4 (restated).

There is a deterministic algorithm that given an undirected multigraph on vertices with normalized Laplacian , a nonnegative integer , and , constructs an undirected multigraph whose normalized Laplacian is an -spectral approximation of . That is, for all vectors

The algorithm runs in space , where is the bit length of the input graph .

The algorithm described below is inspired by techniques used in [CCL2] to approximate random walks with a randomized algorithm in nearly linear time. Our analyses use ideas from the work of Cohen, Kelner, Peebles, Peng, Rao, Sidford, and Vladu on directed Laplacian system solvers even though all of the graphs we work with are undirected.

4.1 Algorithm Description and Proof Overview

Let be the normalized Laplacian of our input and be the target power. We will first describe an algorithm for computing without regard for space complexity and then convert it into a space-efficient approximation algorithm. The algorithm iteratively approximates larger and larger powers of . On a given iteration, we will have computed for some and we use the following operations to increase :

  • Square: ,

  • Plus one: .

Interleaving these two operations appropriately can produce any power of , invoking each operation at most times. To see this, let be the bits of in its binary representation where is the least significant bit and is the most significant. We are given . The algorithm will have iterations and each one will add one more bit from most significant to least significant to the binary representation of the exponent. So after iteration we will have .

For iterations , we read the bits of from to one at a time. On each iteration we start with some power . If the corresponding bit is a 0, we square to create (which adds a 0 to the binary representation of the current exponent) and proceed to the next iteration. If the corresponding bit is a 1, we square and then invoke a plus one operation to produce (which adds a 1 to the binary representation of the current exponent). After iteration we will have .

Implemented recursively, this algorithm has levels of recursion and uses space at each level for the matrix multiplications, where is the bit length of the input graph. This results in total space , which is more than we want to use (cf. Theorem 1.4). We reduce the space complexity by replacing each square and plus one operation with the corresponding derandomized product, discussed in Section 3.

Theorem 3.2 says that the derandomized product produces spectral approximations to the square and the plus one operation. Since we apply these operations repeatedly on successive approximations, we need to maintain our ultimate approximation to a power of . In other words, we need to show that given such that we have:

  1. .

We prove these in Lemmas 4.1 and 4.2. The transitive property of spectral approximation (Proposition 2.2 Part 2) will then complete the proof of spectral approximation.

We only know how to prove items 1 and 2 when is PSD. This is problematic because is not guaranteed to be PSD for arbitrary graphs and so may only be PSD when is even. Simple solutions like adding self loops (to make the random walk lazy) are not available to us because loops may affect the random walk behavior in unpredictable ways. Another attempt would be to replace the plus one operation in the algorithm with a “plus two” operation

  • Plus two: