Two of the most important features of multi-agent systems are their flexibility and scalability. Accordingly, these systems are expected to cope with agent failures and new agent arrivals. Real life examples of the multi-agent systems with such properties include flock of birds, ad-hoc networks of mobile devices, or the Internet. Social systems of various scales have these properties : work teams in companies or laboratories subject to important turnovers, companies themselves, or even entire countries and their cultural norms.
However, the framework typically used to study formal models of multi-agent systems supposes that, while the system may be complex, its composition remains unchanged over time (albeit the interaction topology can evolve). Under this assumption, researchers are able to characterize the long term behavior of the multi-agent system such as convergence and synchronization.
This apparent contradiction is justified when agents arrivals and departures are sufficiently rare as compared to the time-scale of the process taking place in the system. In such cases, it makes indeed sense to assume that the composition of the systems remains unchanged while the process takes place.
Nevertheless, the probability of a node failure is expected to grow with the number of agents. As a consequence, for large systems, this constant size assumption no longer holds. Similarly, in some systems such as living systems with birth processes for instance, the probability of a node arrival increases with the system size, so that the constant composition assumption also stops being relevant when the system size is large. Companies or human societies are instances of such systems where the system’s growth is proportional to its size. This assumption may also appear unsuitable in extreme environments, where communication is difficult and infrequent, leading to slow convergence rate, relative to which the agent failure rate may be important.
Hence we consider here open multi-agent systems, where agents keep arriving and/or leaving during the execution of the process considered, an example of which is illustrated in Figure 1.
Repeated arrivals and departures result in important differences in the analysis or the design of open multi-agent systems and cause several challenges:
State dimension: Every arrival results in an increase of the system state dimension, and every departure in a decrease of the system state dimension. Analyzing the evolution of the system state is therefore much more challenging than in “closed systems”.
Absence of usual “convergence:” Being continuously perturbed by departures and arrivals, open systems will never asymptotically converge to a specific state (this is clear from Figure 1). Rather, they may approach some form of steady state behavior, which can be characterized by some relevant descriptive quantities. As in classical control in the presence of perturbations, the choice of the measures is not neutral, and different descriptive quantities may behave in very different way.
Robustness and quality of the algorithms: Although this is not treated here, departures and arrivals also have a fundamental impact on the design of decentralized algorithms over open systems. These events will often imply a loss of information or a change in the algorithm desired result (information held by new agents may indeed affect a value that the algorithm should compute). Hence the algorithms should be robust to departures and arrivals. On the other hand, algorithms over open systems cannot be expected to be “exact”: When the system composition keeps changing, algorithms able to maintain an approximate answer most of the time may be preferable to those that would eventually provide an exact answer if the system composition were to remain constant.
We provide results which are part of an ongoing study on open multi-agent systems. Here, we study systems where the interactions among agents occur in discrete-time and take the form of an average pairwise gossip algorithm . We assume all-to-all (possible) communications, focusing on systems where departures and arrivals take place at random times, see Section II for a complete definition.
We analyze the system evolution in terms of two “scale-independent” quantities. We find that the first two expected moments form such relevant quantities : namely we study the expected square mean and the expected mean square of the system state
. These quantities also provide the evolution of the expected variance. We show in Section III that these quantities can be characterized exactly, and that they evolve according to an associated 2-dimensional linear system.
In Section IV, we analyze in detail the case of systems with replacements taking place at random time : a departure is immediately followed by an arrival. Each time an event occurs, it is either a replacement or a gossip step with a certain probability. In other words, between two consecutive replacements, gossip steps occur, where
is a non-negative integer random variable. As part of the study, we characterize the variable. We then focus in Section V on growing systems that agent keep joining without ever leaving. It will in particular be shown that random arrivals or replacements can result in a significant performance decrease in terms of variance.
Results on simplified versions of the systems considered here were presented at the Allerton Conference on Communication Control and Computing. The main differences with  are (i) the arrivals and replacements are probabilistic events in this paper, while they followed a deterministic (and generally periodic) sequence in 
, (ii) a study of the system convergence rate and the interpretation of the corresponding eigenvectors, and (iii) a different choice of the moments studied, allowing for simpler proofs.
I-B Other works on open multi-agent systems
The possibility of agents joining or leaving the system has been recognized in computer science, and specific architectures have for example been proposed to deploy large-scale open multi-agent systems, see e.g. THOMAS project . There also exist mechanisms allowing distributed computation processes to cope with the shut down of certain nodes or to take advantage of the arrival of new nodes.
Frameworks similar to open multi-agent arrivals have also been considered in the context of trust and reputation computation, motivated by the need to determine which arriving agents may be considered reliable, see e.g. the model FIRE . However, the study of these algorithms’s behavior is mostly empirical.
Varying compositions were also studied in the context of self-stabilizing population protocols [1, 4], where interacting agents (typically finite-state machines) can undergo temporary or permanent failures, which can respectively represent the replacement or the departure of an agent. The objective in those works is to design algorithms that eventually stabilize on the desired answer if the system composition stops changing, i.e. once the system has become “closed”.
Ii System Description
We consider a multi-agent system whose composition evolves with time. We use integers to label the agents, denote by the set of agents present in the system at time , and by the number of agents present at time , i.e. the cardinality of . Each agent holds a value , and we make no assumptions about the values held at by the agents initially present in the system.
We consider a discrete evolution of the time . It is possible to interpret the discrete time as a sampling of a continuous time variable. Samples then correspond to instants where an event occurred. We will comment later on this interpretation and on its implication on the scaling of different parameters. At each time , one of three events may occur:
(a) Gossip: Two agents are uniformly randomly and independently selected among the agents present in the system (with in particular the possibility of selecting twice the same agent), and they update their values by performing a pairwise average:
(b) Departure: One uniformly randomly selected agent leaves the system, so that and . This event may only occur if .
(c) Arrival: One “new” agent , , joins the system, so that and . The initial value of the arriving agent is drawn independently from a constant distribution with mean and variance . (The results can immediately be adapted to systems where the mean of the arriving agents states is an arbitrary constant).
In addition, we will sometimes consider for simplicity a “replacement” event, which consists of the instantaneous combination of a departure and an arrival: an agent leaves the system and is instantaneously replaced.
Note that all the random events above are assumed independent of each other.
Scale-independent quantities of interest
The aim of the study is to characterize the disagreement among agents, i.e. the distance to consensus. We say that consensus is reached asymptotically when
If the system dynamics does not include agent departures or arrivals, it is known that the gossip process we consider leads to consensus, see e.g.[5, 2]. The objective here is to understand how agent arrivals and departures impact the disagreement among agents. To do so, we study several quantities of interest. Because the system size may change significantly with time, we focus on scale-independent quantities, i.e. quantities whose values is independent of the size of the system. We consider in particular the empirical mean of the squares and the variance defined as
respectively, where references to time were removed to lighten the notation. Our study will focus on the evolution of , which will also require monitoring and . When new agents keep arriving it is impossible to achieve asymptotic consensus in the sense of (2), because the new agent’s value will with high probability be different from the value of the agents already present in the system. The study of will allow us to see how “far” the system will be from consensus. But we will see that in certain systems whose sizes grow unbounded, we may have , corresponding to a form of “almost consensus”. The expected mean could also have been monitored. It evolves following an independent one-dimensional linear system. However, we skip this part of the study due to space limitations.
Iii Effect of different operations
In the sequel we will show that the evolution of the expected moments and is governed by an affine system from which we will derive the evolution of . For notational simplicity we denote by
the vector containingand so that .
Lemma 1 (Gossip step)
Suppose that a randomly selected pair of agents engage in a gossip averaging according to equation (1). Let be the state of the system before that interaction, its state after the interaction and the number of agents. There holds
and as a consequence
Notice that this result or its variations are available in many previous works, but its proof is presented for the sake of completeness.
let us first fix the nodes involved in the gossip. Observe that , and that for all . Hence , which establishes the fist line of (4). For the second line, since for every , there holds
Observe that and . Taking the expectation with respect to and in (III) yields
from which the second line of (4) follows.
Lemma 2 (Arrival of the -th agent)
Suppose that an agent arrives into the system bringing the number of agents from to . Denote the state before arrival, the state after arrival. Then, there holds
We label the arriving agent for simplicity, so that for all . We begin by computing the new average :
Since , we have . By exactly the same reasoning but using we also obtain
Lemma 3 (Departure)
Suppose that a randomly selected agent departs from the system. Denote the state before departure, the state after departure and the number of agents before departure. Then, there holds
Let be the randomly selected agent that leaves the system. The mean is modified as
By exactly the same reasoning, there holds Since is randomly selected, . Hence,
We now consider the replacement of an agent, which consists of a departure immediately followed by an arrival. The next result follows from a combination of Lemma 3 and 2, the latter applied to a system of size joined by a agent.
Lemma 4 (Replacement)
Suppose that a randomly selected agent departs from the system and is immediately replaced by a new agent, leaving the size of the system unchanged. Denote the state before replacement, the state after replacement and the constant number of agents. Then, there holds
Iv Fixed-size system with random replacement
Iv-a Model and fixed points
We assume now that at each time step, there is a probability that an agent is replaced, and a probability that a gossip step takes place. After giving an exact value for the fixed point of the expected system behavior, we will provide asymptotic results when the system size is large. Note that when considering results for large , it is natural to keep constant, although other forms of scaling could be considered. Suppose indeed that our discrete times correspond to the sampling of a continuous-time process at those times at which an event occurs. We suppose that the rates of gossip and of departure (leading to a replacement) of a single agent are both independent of the system size, which would be natural for large systems. As a result, the rates of gossip and replacements at the system level both scale linearly with , so that the probability that a randomly selected event is a replacement remains independent of .
Suppose that an event occurs. This event is an agent replacement with probability or gossip with . Denote the state before the event, the state after. Then, there holds
Since the probability of the events are independent of , the conditional expected value is computed as follows :
Therefore, there holds which yields (13).
One can verify that the fixed point of (13) is
leading to a variance .
The asymptotic values of these expressions admit some interpretation. Suppose first that , meaning that no gossip ever takes place. We obtain then a variance , and an expected square of the average consistently with a process where agents are just replaced, i.e., a system eventually consisting of agents with random i.i.d. values with mean 0 and variance . (This is also the fixed point of the affine equation in Lemma 4). For , the number of gossips steps between two replacements tends to infinity, so that a perfect averaging takes place before any replacement. We obtain in that case a variance , and an expected square average . This latter number is lower than what would be obtained by averaging i.i.d. values. This is because it actually results from a weighted average of the values of all agents having been part of the system the system at some present or past time. See Section IV.c of  for a detailed computation of this value in a system with deterministic replacements.
For large and constant , which we have argued above is a natural scaling, the expected square goes to 0, while the variance goes to . This result is parallel with that obtained in  for periodic replacement, taking into account that the average number of gossip steps between two replacements is . We will come back to this value in Section V.
To illustrate Theorem 5, we consider an open system with random events (replacements or gossip steps) with agents and replacements occur with probability while gossip steps occur with probability . The system has evolved until it has reached replacements. Arriving agent values are drawn uniformly in so that . Figure 2 displays a realization of the trajectories along with the expected dynamics for the scale-invariant quantities. In the top plot, it appears that agents leaving the system (in red) tend to have a more moderate state compared to agents arriving in the system (in blue). This is due to the gossip steps. Despite not leading to a consensus, the open system still presents a contracting tendency. Besides, as seen in the bottom plot, for a sufficient number of agents (here 25), the expected variance rather well approximates the dynamics of empirical variance realization. Also for this number of agents, the square mean remains small (of order ) compared to the mean square, as a consequence, the variance is mainly due to . This would not be the case for agents for instance.
The illustration provided in the introduction (Figure 1) was obtained for an open multi-agent system of the same kind with agents and a replacement probability where the dynamics consider the system up to replacements.
Iv-B Convergence rate
We now study the rate at which the expected moments will converge to the fixed points described above. Eigenvalues of the matrix in (13) were computed on Mathematica :
with (The choice of notation comes from the use of either or in the root expression.) For large and, since , the eigenvalues are of order
which happen to be the diagonal elements of the matrix, as if neglecting the term on the upper right hand side.
The corresponding eigenvectors are
unless in which case higher order term needs to be taken into account.
The couple is independent of for large . The eigenvector has a vanishingly small component in and concerns asymptotically exclusively which is then essentially equivalent to . It follows from Lemma 1 that every gossip iteration will contract this quantity by , and from Lemma 4 that every replacement will also contract that quantity by (when neglecting ) before adding a constant term. If the constant term was set to , would thus contract at a rate at every iteration, independently of whether a gossip or replacement takes place. For large , the couple can thus be related to the contraction of
The couple , on the other hand, does strongly depend on . In particular, if . Remember that the square average remains unchanged when gossip iterations occur. But when a replacement occurs, it follows from Lemma 4 that the new square average contains a contribution of the previous value, some constant contribution related to the arriving agent, and a vanishingly small contribution related to the average square value (assuming that and are of the same order of magnitude, which is the case in when ). Hence, if we were to set the independent term at 0 in (12) corresponding to , the square average would be multiplied by at each replacement and by 1 otherwise. Since replacement occur with a probability , this yields a rate .
Observe now that there is a transition at . Indeed, for , corresponding to frequent replacements, the largest eigenvalue is . The rate of convergence to the steady state depends thus on the rate of variance reduction, which is independent of (for large ). On the other hand, for , the convergence is generally dominated by phenomena related to the convergence of the the square average, which does depend on as this quantity only changes when a replacement takes place.
V Growing system without departure
We focus now on systems whose sizes grow unbounded because new agents keep joining while no agent ever leaves: Similarly to the fixed-size model described in Section IV-A, at each time, a new agent joins the system with probability or a gossip step occurs with probability , where is the number of agent in the system. The discussion of the dependence of on is deferred to the end of the section. When no ambiguity is possible, we will simply use . We assume again that the initial value of every agent when joining the system is a random variable with zero mean and variance . We denote by the time just after the arrival of the -th agent, and we let be the number of gossip steps taking place between the arrival of agents and . Both the and are random variables, and the
follow a geometric distribution with parameterso that . Since we are only interested in expected quantities and that the set of sequences which are bounded has probability , we assume in the sequel that sequence is unbounded.
We will focus on the values of the expected average of the square and the expected square of the average at the times , just after the arrivals of the -th agents. We will see that the evolution of these values can be described by a two-dimensional linear system. This system is here time-varying because is not constant, but the absence of departures makes it triangular, and hence easier to analyze. We first provide the evolution of the expectation of vector between two agent arrivals.
Let be the state of the system after the arrival of the agent, and suppose that at every time-step, an arrival takes place with probability and a gossip with probability . Let then be the state after the arrival of agent , so after the first arrival event. There holds where
To obtain the expectation of vector after the gossip, we condition with regards to which is by definition the number of gossip steps having taken place. By Lemma 1, there holds
Recalling that , the expectation of the previous equation provides the result. A consequence of this lemma is that between two agent arrivals, the expected variance evolves autonomously as
Applying Lemma 2, we obtain the evolution of the variance after agent arrivals.
Recall that is the state of the system just after the arrival of the -th agent and the state just after the arrival of the -th agent, so that is obtained starting from state and applying gossip steps and an arrival. Then, there holds
Since is diagonal and is lower triangular, the first line of the previous equation provides
We conclude using equation (16).
This recursion allows obtaining the following theorem characterizing the asymptotic variance, and proved in Appendix -A.
Consider the growing system without departure, and remember that when the system with agents undergoes an event, describes the probability of the -th agent arrival rather than a gossip step.
(i) If for all for some , then
(ii) If , then .
Theorem 8(ii) shows that the system essentially converges to a consensus as soon as goes to zero, even if this convergence is very slow. This can also be interpreted in terms of , the expected number of gossip steps between two arrivals. Since follows a geometric law of parameter , which diverges to when goes to zero. As a consequence, the system converges to a consensus as soon as the expected number of gossip steps between two consecutive arrivals diverges, and even if the number of gossips per agent between two consecutive arrivals tends to 0. Note, however, that each agent gets involved (with probability 1) in infinitely many gossips when . The expected number of gossips in which an agent has been involved at time is indeed , which grows unbounded.
By contrast, in the case of a fixed probability corresponding to a fixed with , agents have on average been involved in gossips after any given arrival, which intuitively explains why the variance stays bounded away from 0. But the actual asymptotic value obtained in Theorem 8(i) is remarkably high, and is actually the same as that obtained for the fixed-size system in Section IV-A for large . As a basis for comparison, suppose we had first waited until the independent agents were present in the system, which would yield an expected variance , and then performed the same number of gossip averaging operations between randomly selected pairs of nodes. It follows from application of equation (5) that the expected variance would then have been
which is significantly lower than (for , the ratio of variance would be ). The dynamics of the system composition deteriorates thus considerably the performances in terms of variance reduction.
We now propose possible interpretations of the evolutions of and with the number of agents . Suppose that we interpret our discrete as the sampling of a real continuous time variable at those times at which an event occurs. It is again reasonable to assume the interaction rate of an agent to be independent of the system size, so that the total number of gossips per unit of time would grow linearly with , as say . Suppose first that the agents arrive at a fixed rate . In that case, the probability of agent arrival is and the number of gossips between two arrivals would be linearly growing with and . Theorem 8(ii) shows then that the variance would converge to 0.
But one could also imagine a linearly growing rate of arrivals . This would for example be the case if the system attraction were growing with its size or if the arrivals resulted from some form of reproduction process. The probability of agent arrival would then be constant and so will be the number of gossip iterations between two arrivals , leading to a finite variance .
We have made first steps in the analysis of open multi-agent systems, where agents can leave and arrive. We have focused on analysing open systems subject to a classical multi-agent algorithm : on all-to-all pairwise gossips. We have shown that these systems could be characterized by fixed-size linear systems in terms of some of the moments. Interestingly, we have also observed that the open character of the system may result in a significant performance reduction in terms of variance reduction.
Ongoing works include the generalization of this approach to systems where arrivals and departure follow more complex patterns, or to more complex interactions, such as gossips restricted to a graph.
Another challenge left untackled so far is the characterization of the variability for individual realizations. Our results characterize for instance the expected value of the disagreement in the system where
, but do not directly allow deducing the width of the probability distribution of.
-  Dana Angluin, James Aspnes, Michael J Fischer, and Hong Jiang. Self-stabilizing population protocols. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 3(4):13, 2008.
-  Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Randomized gossip algorithms. IEEE/ACM Transactions on Networking (TON), 14(SI):2508–2530, 2006.
-  Carlos Carrascosa, Adriana Giret, Vicente Julian, Miguel Rebollo, Estefania Argente, and V Botti. Service oriented mas: an open architecture. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 1291–1292. International Foundation for Autonomous Agents and Multiagent Systems, 2009.
-  Carole Delporte-Gallet, Hugues Fauconnier, Rachid Guerraoui, and Eric Ruppert. When birds die: Making population protocols fault-tolerant. In International Conference on Distributed Computing in Sensor Systems, pages 51–66. Springer, 2006.
-  Fabio Fagnani and Sandro Zampieri. Randomized consensus algorithms over large scale networks. IEEE Journal on Selected Areas in Communications, 26(4):634–649, May 2008.
-  Julien M. Hendrickx and Samuel Martin. Open multi-agent systems: Gossiping with deterministic arrivals and departures. 54th Annual Allerton Conference on Communication, Control, and Computing, 2016.
-  Trung Dong Huynh, Nicholas R Jennings, and Nigel R Shadbolt. An integrated trust and reputation model for open multi-agent systems. Autonomous Agents and Multi-Agent Systems, 13(2):119–154, 2006.
Gerardo Iñiguez, János Török, Taha Yasseri, Kimmo Kaski, and
Modeling social dynamics in a collaborative environment.
EPJ Data Science, 3(1):1, 2014.
-  János Török, Gerardo Iñiguez, Taha Yasseri, Maxi San Miguel, Kimmo Kaski, and János Kertész. Opinions, conflicts, and consensus: modeling social dynamics in a collaborative environment. Physical review letters, 110(8):088701, 2013.