Swarm robotics focuses on the study of distributed robotics systems composed of a large number of independent and autonomous robots. Swarm engineers design these systems with three objectives in mind: i. scalability, the ability of the system to keep functioning for an increasing number of components; ii. flexibility, the ability of the system to adapt to unknown or changing environmental conditions; and iii. robustness, the ability of the system to undergo a graceful degradation of its performance due to the loss or malfunctioning of one or more of its components (brambilla|ferrante|birattari|dorigo:2013; dorigo|birattari|brambilla:2014). The lack of a centralized authority governing the functioning of a robot swarm is one (among others, cf. brambilla|ferrante|birattari|dorigo:2013) of the fundamental pillars that allow engineers to design robotic systems that can meet these objectives. However, when no central authority is present, the swarm requires mechanisms that allow its numerous members to make decisions collectively (valentini|ferrante|dorigo:2017; valentini:2017).
A collective decision is the result of a process distributed among a collective of agents that leads the collective to make a choice that, once made, can no longer be traced back to any of its individual agents. Collective decisions are widespread in group-living animals (conradt2003group; sumpter:2006) and are studied across different scientific disciplines such as psychology (moscovici|etal:1969; hirokawa|poole:1996), biology (camazine|deneubourg|franks|sneyd|theraulaz|bonabeau:2001; couzin|ioannou|demirel|gross|torney|hartnett|conradt|levin|leonard:2011; conradt|list:2009), and physics (castellano|fortunato|loreto:2009; galam:2008), to name a few. Examples of collective decisions include the ways with which eusocial insects, such as ants and bees, explore a large portion of their environment, identify several candidate locations, and eventually choose a single, possibly optimal option where to move and create a new home (franks|pratt|mallon|britton|sumpter:2002; Visscher:2007; McGlynn:2012); the mechanisms used by groups of vertebrates, such as bird flocks, fish schools, or primate troops, to coherently and suddenly change direction of motion in response to predators or other sources of external information (Okubo:1986; couzin|krause|franks|levin:2005; sumpter:2010; CONRADT2010675; kao|miller|torney|hartnett|couzin:2014; Strandburg-Peshkin|etal:2015); even organisms that lack a brain, such as slime molds, are sometimes able to perform complex computation and make collective decisions (reid:2016; ray|etal:2019). In these examples, the collective decision is represented by a consensus shared by all (or a large majority) of the members of the collective.
Not all collective decision-making processes, however, result in a consensus among the members of the collective. Some of these processes require the collective to allocate its members to a number of different tasks (or options) in a manner that optimises performance. For example, in the realm of eusocial insects, workers take care of a number of different tasks, including foraging, brood care, and nest construction, and change their allocation as a function of colony needs, seasonality, or circadian rhythm (wiernasz2008mating; jandt|gordon2016; gordon2016; Charbonneau2015; doi:10.1098/rstb.2017.0235). Studies of task allocation and, more generally, division of labor led to the development of different models (beshers2001models) and ultimately inspired the design of artificial systems in different sub-fields of engineering (johnson2002emergence; gerkey|mataric:2004).
In the context of the design of robot swarms, brambilla|ferrante|birattari|dorigo:2013 distinguished these two scenarios as different categories of collective decisions (Figure 1). Consensus achievement encompasses problems that require the members of a swarm to agree on a single choice (valentini:2017). Task allocation includes instead problems that require the members of the swarm to allocate themselves to different tasks and, in so doing, to break down a bigger problem into a set of smaller ones that can be tackled in a manner that optimizes the performance of the swarm (gerkey|mataric:2004). In this article, we focus on consensus achievement problems. This type of problems can be further divided into two subcategories (Figure 1): problems where the swarm needs to choose one among a discrete and finite set of alternatives, i.e., to make a discrete choice; and problems where instead the number of options is uncountable and the choice is continuous such as the direction of motion of a swarm of agents (reynolds:1987; Olfati-Saber|Fax|Murray:2007).
While this paper focuses on discrete consensus achievement problems in the context of robot swarms, it is worth noting that discrete and continuous consensus achievement are the subject of studies in other branches of engineering. In artificial intelligence and robotics, for example, discrete problems are considered when agents in a team need to cooperate with each other(pynadath|tambe:2002; bernstein|etal:2002) such as in the context of the RoboCup competition (kitano|etal:1997) where both centralized and decentralized solutions are studied (bowling|bowling|veloso:2004; kok|etal:2003a; kok|etal2003b); or in the context of sensor fusion where the solution to similar problems is required to perform distributed object classification (kornienko|kornienko|constantinescu|pradier|levi:2005; kornienko|kornienko|levi:2005). Continuous problems, which are not dealt with in this paper, are also widespread in swarm robotics both in the context of autonomous ground robots (turgut|etal:2008; nembrini|etal:2002; spears|etal:2004; ferrante|etal:2012; ferrante|etal:2014) and in that of unmanned aerial vehicles (holland|etal:2005; huert|etal:2011). The community of control theory has also intensively studied the problem of continuous consensus achievement (mesbahi|egerstedt:2010) for formation control (ren|beard|atkins:2005), agreement on state variables (hatano|mesbahi:2005), sensor fusion (ren|beard:2008), and the selection of motion trajectories (sartoretti|hongler|deoliveira|mondada:2014). Other engineering applications that can be regarded as continuous consensus achievement have been explored for the design of wireless sensor networks (schizas|giannakis|roumeliotis|ribeiro:2008; schizas|ribeiro|giannakis:2008) and other network scenarios (Komareji|Bouffanais:2013; shang|bouffanais:2014). Although related to the studies presented here, the solutions devised in other branches of engineering tend to be too complex to be directly implemented on robot swarms due to high communication requirements.
In the rest of this paper, we focus our discussion on discrete consensus achievement problems. We review the best-of- problem (valentini|ferrante|dorigo:2017), a formalization of problems requiring discrete consensus achievement, and discuss in detail all of its possible variants. We then discuss three examples of application scenarios that can be modeled using this framework: the shortest-path problem, site-selection, and collective perception. Finally, we review a design framework that allows designers to develop collective decision-making strategies that are amenable to mathematical modeling.
2 The Best-of- Problem
In the swarm robotics literature, a large number of research studies focused on a relatively few application scenarios whose accomplishment requires the swarm to solve a consensus achievement problem (e.g., the shortest-path problem in foraging scenarios, site-selection in aggregation scenarios). These application scenarios have been primarily tackled separately from each other with an application-oriented perspective that resulted in either the development of domain-specific methodologies or the design of black-box controllers (valentini|ferrante|dorigo:2017). However, as we show in the rest of this paper, the consensus achievement problems underlying these application scenarios share a similar rationale and structure and it is possible to unify them into a unique framework: the best-of- problem.
The best-of- problem represents any decision-making problem in which a swarm of robots needs to make a choice for one option among a finite set of different alternatives. Robots in the swarm have, or acquire over time, an opinion about which of the available options represents the most beneficial choice for the swarm. This opinion usually changes over time as the robots in the swarm gather and process information about the problem at hand. The solution of the best-of- problem, i.e., a collective decision, is represented by a large majority of the robots in the swarm sharing the same opinion. In order to maintain cohesion of the swarm, it is particularly important that the portion of the swarm agreeing on the same option is as high as possible otherwise the swarm may incur fragmentation. Consider for example an emigration problem in which options correspond to different locations in the environment. If the members of the swarm are spread over different alternatives, some of them might get lost once the emigration towards the location favored by the majority begins. In the extreme case in which all members of the swarm agree on the same option, we say that the swarm has reached a consensus decision.
In general, different options provide the swarm with different benefits and come at different costs. Each option is therefore described by two characteristics: a quality and a cost. Both of them, in turn, can be a function of several attributes that depend on the specific scenario faced by the swarm (reid|garnier|beekman|latty:2015). We borrow an example from Nature to further explain these concepts. When the reproductive season for honeybees arrives, the swarm splits and roughly half of its members emigrate to a new nest site (seeley2010honeybee). Honeybees have a preference for sites with a certain volume, exposure to the sun, and height from the ground (camazine|visscher|finley|vetter:1999). These attributes, possibly weighted in different ways, contribute to the perception of the site quality by individual bees. The distance of the new nest also affects the decision of the swarm as sites that are too far might be impossible to find by individual bees and sites that are too close to the original nest might result in attrition between different swarms. In this example, distance represents therefore the cost of an option while a weighted combination of the site’s attributes represents its quality.
In the context of swarm robotics, the quality and the cost of each option depend on the specific target scenario and on the choices of the designer. Furthermore, depending on the ways with which quality and cost interact with each other, the definition of the optimal solution of the best-of- problem can differ (Figure 2). Both quality and cost can be either symmetric, i.e., all options are equivalent to each other, or asymmetric, i.e., options differ in quality and/or cost. Moreover, the interaction between quality and cost for each option can be either synergistic, i.e., producing a greater combined result, or antagonistic, i.e., playing against each other. The combinations of these proprieties define five possible variants of the best-of- problem with different concepts of which option is the best:
All options have the same quality and the same cost and are therefore equivalent (also know as a symmetry-breaking problem (hamann|schmickl|worn|crailsheim:2012; devries|biesmeijer:2002));
All options have the same quality but at least two of them have different cost, in which case one of the options with minimum cost is to be preferred;
All options have the same cost but at least two of them have different quality, in which case one of the options with maximum quality is to be preferred;
Options can have different quality and different cost but the interaction between these characteristic is synergistic and the best option has maximum quality and minimum cost;
Options quality and costs differ but this time their interaction is antagonistic so that there is not a unique option to be preferred but a trade-off between different design choices.
Moreover, the quality and/or the cost of the available options can either be static or vary over time and be therefore dynamic. Static problems, which so far have been the primary subject of swarm robotics research, are usually addressed using collective decision-making strategies that result in consensus decisions (parker|zhang:2009; montesdeoca|ferrante|scheidler|pinciroli|birattari|dorigo:2011; valentini|birattari|dorigo:2013; valentini|hamann|dorigo:2014; valentini|hamann|dorigo:2015c; scheidler|brutschy|ferrante|dorigo:2015). Dynamic problems are instead less explored in the literature of swarm robotics. In this case, to allow for the continuous exploration of new options or changes in quality and/or cost of previously discovered options, designers favor strategies that do not converge to a consensus but leave a minority of the members of the swarm unaligned with the majority (arvin|turgut|bazyari|arikan|bellotto|yue:2014; parker|zhang:2010; Prasetyo2019; CollectiveChangeDetection).
3 Possible application scenarios
As introduced above, the best-of- problem is an abstraction that can be cast to a large number of different application scenarios. While we refer the reader to (valentini|ferrante|dorigo:2017) for a deeper discussion of this topic, we review here three example scenarios: shortest-path, site-selection, and collective perception. For the sake of simplicity, we present these scenarios in their binary form which can be modeled as a best-of-2 problem.
In its simplest form, the shortest-path problem is represented by a scenario in which a source area and a destination area can be reached by traversing one of two possible paths, i.e., the options of a best-of-2 problem (Figure 3a). Robots in the swarm need to transport resources from the source area to the destination area. As both paths allow robots to successfully transport resources between areas, their quality is equal and symmetric. However, the efficiency with which a robot can do so might vary depending on environmental factors such as the length of a path or its roughness which affect the cost of each option. Unless options have the same cost, in which case they are all equally good, the optimal solution of the shortest-path problem is represented by the path of minimum cost. Several strategies have been developed over the years to address this problem and generally leverage the fact that the shortest path is the faster to traverse to bias the collective decision-making process (campo|etal:2010; montesdeoca|ferrante|scheidler|pinciroli|birattari|dorigo:2011; garnier|combe|jost|theraulaz:2013; valentini|birattari|dorigo:2013; scheidler|brutschy|ferrante|dorigo:2015; reina|valentini|fernandezoto|dorigo|trianni:2015; Valentini2018).
Another particularly popular application scenario studied in the context of the best-of- problem is site-selection (garnier|gautrais|asadpour|jost|theraulaz:2009; schmickl|thenius|moeslinger|radspieler|kernbach|szymanski|crailsheim:2009; parker|zhang:2011; valentini|hamann|dorigo:2014; valentini|hamann|dorigo:2015c; valentini|ferrante|hamann|dorigo:2016; reina|valentini|fernandezoto|dorigo|trianni:2015). In this scenario, the swarm is located in an environment characterized by two or more sites which are of interest for some reason (e.g., the swarm may be required to choose a construction site). Sites are characterized by a quality. In the scenario depicted in Figure 3b, for example, the quality of a site might be proportional to its area in which case the best site would correspond to area 1. Sites can also have an associated cost: assuming the swarm is randomly exploring the environment, area 1, being larger, is easier to discover than area 2. In this scenario, designers favor therefore strategies than can both exploit environmental biases (i.e., those associated with the cost of each option) and that can introduce a controlled bias (i.e., a mechanism that is intrinsically coded into the robot behavior) modulated by the robot’s perception of the quality of a site.
Finally, an application scenario that has been recently introduced in the context of swarm robotics is that of collective perception (valentini|brambilla|hamann|dorigo:2016). In a collective perception scenario, a swarm of robots is located in an environment characterized by different features and has to decide which feature is the most spread. In the example depicted in Figure 3
c, features are represented by the colors white and blue. In a real application, features may correspond to different resources available in the environment and the swarm would therefore be tasked with classifying the environment as a reservoir of the most spread resource. While the abundance of each feature represents the quality of that particular option, options could also have an associated cost (e.g., the time necessary to mine or collect a particular resource). After its introduction by(valentini|brambilla|hamann|dorigo:2016), the collective perception scenario has been recently considered in other studies as a benchmark for collective decision making (Strobel:2018; ebert|gauci|nagpal:2018).
4 A design framework
problem can be address using strategies designed following different approaches (e.g., evolutionary computation). In this section, we review a behavior-based approach put forward in(valentini:2017) which aims at designing simple strategies whose dynamics are amenable to mathematical modeling. This characteristic is of particular relevance to swarm engineers as mathematical models allow designers to extensively study the performance of their strategies under different assumptions without relying on time-consuming simulations or robot experiments. Such collective decision-making strategies can be designed by combining a few key components that suffice to address all five variants of the best-of- problem. These components are: i. a mechanism for exploration of options, ii. one for the dissemination of opinions, iii. and a decision rule that allows robots to change their opinion.
Robots in the swarms need a mechanism to explore the environment. This mechanism is necessary to discover possible alternatives of the best-of-
problem as well as to gather sample estimates of the associated quality. Depending on the target application scenario, exploration can be implemented as a simple random search of the environment or can combine more complex search strategies that rely on other factors, for example, that leverage the presence of a light source to drive the motion of robots using phototaxis(valentini|ferrante|hamann|dorigo:2016). The duration of the exploration phase is generally affected by environmental factors that are beyond the reach of the designer. This point is of particular relevance as the cost of each option is generally “paid” in terms of the time spent exploring that option which affects the frequency with which robots can disseminate their opinion and influence the decision-making process.
Once a robot has sampled the quality of an option, it is ready to disseminate its opinion about that option within the rest of the swarm. To do so, it is important that the swarm has some sort of central location (e.g., a deployment area) where robots can exchange their opinions. Disseminating an opinion can be as simple as broadcasting it locally towards other members of the swarm. The duration of the dissemination phase is controlled by the designer and, in order to influence the collective decision towards a consensus for the best option, it needs to be proportional to the quality of the option being advertised. By doing so, options with higher quality are advertised for longer time and have higher chances to be heard by other members of the swarm. Indeed, during the dissemination phase, a robot also listens to the opinions of other robots and keeps track of their frequency.
Before terminating a dissemination period, a robot applies a decision rule to reconsider and possibly change its own opinion. A decision rule is a function that takes as input the current robot opinion and the set of opinions perceived from neighboring robots and outputs a new opinion. To minimize the side-effects of delayed applications of a decision rule, the set of opinions of neighboring robots should be formed only by the most recently perceived opinions. Failure to do so might result in a robot applying a decision rule using outdated information possibly impacting the collective decision-making process. Popular decision rules are represented by the voter model, with which a robot adopts the opinion of a randomly chosen neighbor, and the majority rule, with which a robot adopts the opinion shared by the majority of its neighbors (valentini|ferrante|hamann|dorigo:2016). Other decision rules are possible but an extensive discussion of this point is beyond our scope (see (valentini|ferrante|dorigo:2017) for more information).
Finally, all these mechanisms are combined using a finite state machine as illustrated in Figure 4. From an exploration state, E, in which the robot sampled the quality of option , it will deterministically transition into the dissemination state D associated to the corresponding opinion. After disseminating opinion for a time proportional to its quality, the robot applies a decision rule that is function of the most recently perceived opinions of its neighbors. The outcome of the decision rule, which depends on the composition of its neighborhood, will determine which option the robot will explore next. The execution of this finite state machine continues until the swarm reaches a collective decision. Note that repeated explorations of the the same options, even if a robot does not change opinion after applying the decision rule, are required to minimize the impact of noise during sampling of the associated quality.
The ability to make decisions collectively is a fundamental pillar for the development of autonomous robot swarms. In this paper, we reviewed the best-of- problem, a formal framework to model discrete consensus achievement problems for robot swarms. We showed that this framework can be used to model different application scenarios. The advantage of taking this perspective is that, once a particular strategy to address the best-of- problem is in place, it can be easily re-implemented to address a different problem scenario as showed in (valentini|brambilla|hamann|dorigo:2016). We reviewed a design framework to devise behavior-based strategies that are amenable to mathematical modeling. This design framework is appealing to swarm engineers seeking to guarantee the performance of their distributed algorithms through formal analysis of the resulting population dynamics (valentini:2017) and/or information transfer processes (valentini|etal:2018alife). Simple collective decision-making strategies can be later extended to incorporate more advanced mechanisms and improve performance and flexibility (talamali2019improving).