Within the Multi-agent System (MAS) area, norms are understood as means to coordinate and regulate the activity of autonomous agents interacting in a given social context . The existence of autonomous agents that are capable of violating norms entails the development of norm control mechanisms that implement norms in agent societies.
In the existing literature, several authors have proposed infrastructures to observe agent actions and detect norm violations upon them [1, 21]. The majority of these proposals have focused on providing efficient and scalable methods to monitor norms in dynamic agent societies, but they assume that all actions of agents are observable. However, this assumption is too strong because it is not necessarily true that all actions to be controlled can always be observed. One reason for this is that observing actions usually entails high costs. For example, the costs of setting, maintaining, and managing traffics radars to detect car speeds are very high, so traffic authorities usually decide to install a few of them in specific and critical locations. Another reason is that illegal actions may take place outside the institution controlled by the monitor; however, the effects of these actions can still be detected within the institution. For example, black market transactions cannot be directly observed by legal authorities, yet the corresponding money laundering transactions can be detected and sanctioned by these authorities.
Very recent work on norm monitoring under partial action observability proposes solutions to ensure complete action observability by increasing the actions that are observed, either by adding more monitors  or by adapting the norms to what can be observed . However, these solutions are not always appropriate or feasible. For instance, in e-markets, such as eBAY111http://www.ebay.com or Amazon222http://www.amazon.com, it is not possible to change trading laws to what can be observed. This paper goes beyond these approaches by also considering actions that were not observed but that can be reconstructed from what was observed.
The main contributions of this paper are: (i) a formalisation of the problem of reconstructing unobserved actions from observed actions for the purpose of norm monitoring; (ii) an exhaustive and an approximation solution to this problem; and (iii) an information model and algorithms used to monitor norms under partial action observability. Through an extensive empirical evaluation, we show that reconstructing unobserved actions increases noticeably the number of norm violations and fulfilments detected.
This paper is organised as follows: Section II contains the preliminary definitions used in this paper. Section III describes the information model of norm monitor proposed in this paper. Section IV contains the algorithms executed by norm monitors. Our proposal is evaluated in SectionV. Related word is discussed in Section VI. Finally, conclusions are contained in Section VII.
Ii Preliminary Definitions
is a first-order language containing a finite set of predicate and constant symbols, the logical connective , the equality (inequality) symbol (), the true () and false propositions (), and an infinite set of variables. The predicate and constant symbols are written as any sequence of alphanumeric characters beginning with a lower case letter. Variables are written as any sequence of alphanumeric characters beginning with a capital letter. We also assume the standard notion of substitution of variables ; i.e., a substitution is a finite and possibly empty set of pairs where is a variable and is a term.
The set of grounded atomic formulas of
is built of a finite set of predicates and objects that characterise the properties of the world relevant to norm monitoring. By a situation, we mean the properties that are true at a particular moment. Some of these properties are static and not altered by action execution, whereas other properties are dynamic and changed due to agent actions. Specifically, we represent static properties as a set333In this paper sets are to be interpreted as the conjunction of their elements. of atomic grounded formulas of , denoted by . A state is a set of grounded atomic formulas of , describing dynamic properties which hold on state . Thus, a situation is built on a “closed assumption” and defined by a set of static properties and a state . Moreover, there is a set of inference rules () representing domain knowledge.
Example. In this paper we will use a running example in which there are three robots that should attend requests at six offices in a building. The goal of the robots is to attend these requests as soon as possible. Figure 0(a) depicts our initial scenario. In our example, the language contains: 4 predicate symbols (), used to represent the robots and offices, the positions of the robots and the connections between offices in the building; 3 constant symbols to represent the robots (); and 6 constant symbols to represent the offices (). The information about the robots, offices and corridors between offices is static and represented as follows:
The information about the location of the robots is dynamic. Specifically, the initial state is defined as follows:
In this domain there is an inference rule () representing that a robot cannot be in two different offices at the same time:
Ii-a Action Definitions
is a finite set of action descriptions that induce state transitions. An action description is represented using preconditions and postconditions. If a situation does not satisfy the preconditions, then the action cannot be applied in this situation. In contrast, if the preconditions are satisfied, then the action can be applied transforming the current state into a new state in which all negative literals appearing in the postconditions are deleted and all positive literals in the postconditions are added. Moreover, actions are executed in a MAS and, as a consequence, we need to be able to represent concurrent actions with interacting effects. For the sake of simplicity, we will represent concurrent actions without an explicit representation of time444An explicit representation of time may play a role on other problems like scheduling concurrent actions, but is not strictly necessary for monitoring the effects of interaction. as proposed in . The main idea beyond this representation is that individual agent actions do interact (i.e., one action might only achieve the intended effect if another action is executed concurrently). Specifically, each action is also represented by a (possibly empty) concurrent condition that describes the actions that must (or cannot) be executed concurrently555A more sophisticated definition of the concurrent condition would allow actions to have conditional effects according to the actions that are executed concurrently. Without loss of expressiveness, we will not consider conditional effects in action descriptions (note that any action with conditional effects can be represented by a set of actions with non conditional effects)..
An action description is a tuple where:
is the action name;
is the precondition, i.e., a set of positive and negative literals of (containing both dynamic and static properties) as well as equality and inequality constraints on the variables;
is the concurrent condition; i.e., a set of positive and negative action schemata666An action schema contains an action name and the parameters of this action. Note that positive action schemata are implicitly existentially quantified –i.e., one instance of each positive schema must occur concurrently– and negative schemata are implicitly universally quantified., some of which can be partially instantiated or constrained;
is the postcondition; i.e., a set of positive and negative literals of (containing dynamic properties only).
Given an action description , we denote by the action precondition, concurrent condition and postcondition.
Example. In our example, there is only one action that can be executed by robots:
This action represents the movement of a robot from one office to another. The parameters of this action are the robot (), the source office (), the destination office (). To execute this action, the robot should be located at the source office and the two offices should be connected. Once the operation has been applied, the robot is no longer at the source office and it is at the destination office.
Given a situation represented by the state and a set of static properties , and an action description ; an action instance (or action) is a tuple such that:
There is a substitution of variables in , such that the precondition is satisfied (i.e., entalied by) the situation; i.e., ;
is a set of grounded literals in containing dynamic properties only;
Given an action , we denote by the agent performing the action, and by the precondition, concurrent condition and postcondition.
Example. In state , the robot moves from office to office . This is formalised as follows:
In a MAS, concurrent actions777Concurrent action means actions that occur at the same time and does not necessarily imply agent cooperation or coordination. define state transitions. More formally, a concurrent action is a set of individual actions. Given a set of actions , we define , and .
Given a concurrent action we say that the concurrent condition of an individual action of is satisfied when for all positive schema in the concurrent condition exists an action () in , such that is an instance of the schema; and for all negative schema none of the elements in is an instance of the schema. For the sake of simplicity, we assume that each agent performs one action at a time888This limitation can be relaxed by decomposing agents into groups of agents corresponding to agents’ actuators ..
(Consistency ) Given a concurrent action it is consistent if:
is consistent (i.e, );
is consistent (i.e, );
the concurrent condition of each action is satisfied;
the concurrent action is complete (i.e., each agent performs one action in ).
Example. The concurrent action 999For simplicity, we represent actions by their schemata. is consistent since:
which is consistent;
which is consistent;
the concurrent conditions of both actions are satisfied;
each robot performs one action.
A concurrent action is applicable in a situation if is consistent and each individual action is applicable in this situation.
Given a consistent action, we define its effects as the postconditions of its individual actions and the preconditions not invalidated by the postconditions. More formally, given a concurrent action its effects are a set of grounded literals as follows:
Ii-B Norm Definitions
We consider norms as formal statements that define patterns of behaviour by means of deontic modalities (i.e., obligations and prohibitions). Specifically, our proposal is based on the notion of norm as a conditional rule of behaviour that defines under which circumstances a pattern of behaviour becomes relevant and must be fulfilled [21, 29, 19, 11].
A norm is defined as a tuple
is the deontic modality of the norm, determining if the norm is an obligation () or prohibition ();
is a set of literals of as well as equality and inequality constraints that represents the norm condition, i.e., it denotes the situations in which the norm is relevant.
is a positive action schema that represents the action controlled by the norm.
Example. In our example, there is a norm that avoids collisions by forbidding any robot to move into an office when the office is occupied by another robot:
This norm states that when a robot is located in office other robots are forbidden to move from any office to .
In line with related literature [1, 20, 2], we consider a closed legal system, where everything is considered permitted by default, and obligation and prohibition norms define exceptions to this default permission rule. We also define that a norm is relevant to a specific situation if the norm condition is satisfied in the situation. Besides, we define that a norm condition is satisfied in a given situation when there is a substitution of the variables in the norm condition such that the constraints in the norm condition are satisfied and the positive (vs. negative) literals in the norm condition are true (vs. false) in the situation.
Given a specific situation denoted by a state and a set of static properties , and a norm
; a norm instance is a tuple such as:
There is a substitution such that the condition is satisfied in the situation; i.e., ;
Example. In state the norm that forbids robots to move into occupied offices is instantiated as follows:
The semantics of instances (and norms in general) depends on their deontic modality. An obligation instance is fulfilled when the mandatory action is performed and violated otherwise, while a prohibition instance is violated when the forbidden action is performed and fulfilled otherwise. We classify detected violations (vs. fulfilments) into:identified violations (vs. fulfilment), which refers to when the monitor knows the specific action that an agent executed and violates (vs. fulfils) an instance; and discovered violations (vs. fulfilment), which refers to when the monitor knows that an agent violated (vs. fulfilment) some instance but does not know the forbidden (vs. mandatory) action executed by the agent.
Iii NM Information Model
Let us assume a set of agents to be monitored, a set of norms that regulate the actions of agents, and a set of action descriptions that represent the actions that can be performed by agents. For the sake of simplicity, we assume that there is a single Norm Monitor (NM) that observes the actions performed by agents and monitors norm compliance101010However, our model can be used by a team of monitors as well.. We also assume that actions are deterministic and that the current state evolves due to action execution only111111This assumption could be relaxed if NMs have capabilities for observing both state changes and actions.. The goal of the NM is to analyse a partial sequence of action observations to detect norm violations. The enforcement of norms is out of the scope of this work and we assume that once the NM detects a norm violation (vs. fulfilment), it applies the corresponding sanction (vs. reward).
Iii-a State Representation
As the NM only observes a subset of the actions performed by agents, it has partial information about the state of the world. The NM represents each partial state of the world, denoted by , using an “open world assumption” as a set of grounded literals that are known in the state. Thus, a partial state contains positive (vs. negative) grounded literals representing dynamic properties known to be true (vs. false) in the state. The rest of dynamic properties are unknown.
To begin with, assume that the NM monitor has complete knowledge of the initial state (this will be relaxed later). Thus, at the NM knows which grounded atomic formulas are true or false in the initial state (). From that moment on, the NM monitors the actions performed by agents at each point in time. At time the NM carries out a monitoring activity and observes some of the actions performed by agents (). These actions have evolved into a new state . As previously mentioned, the NM has limited capabilities for observing the actions performed by agents. Thus, it is possible that the NM observes a subset of the actions performed by agents. Specifically, if all actions have been observed (), then the resulting partial state can be constructed by considering the effects of actions in on so . A different case arises when the NM observes a subset of the actions performed by the agents (). In this case, the agent cannot be sure about the effects of unobserved actions. Thus, the new partial state is constructed by assuming that the postconditions of the observed actions must hold on state (i.e., positive postconditions are positive literals in and negative postconditions are negative literals in ) and the rest of dynamic propositions are unknown. If the NM takes into account the next sequence of actions that it observes at time (), then the NM can also infer that the preconditions of these actions must hold on state , and, as a consequence, new propositions can be taken for sure in the partial state , retrospectively. Partial states in the general case are defined as:
Given a partial state description corresponding to time , and two consecutive sequences of observed actions and executed by agents at times and , respectively; the new partial state resulting from executing actions in and actions in is obtained as follows:
where is the set of invariant literals; i.e., literals of that have not been modified by the actions in and it is defined as follows:
Example. In our example, the NM knows which grounded atomic formulas are true or false in the initial state:
The NM has some surveillance cameras to monitor the movement of robots in the building. Specifically, the corridors that are monitored are the ones between offices: and ; and ; and and . These corridors are represented by black arrows in Figure 1, whereas non-monitored corridors are represented by grey arrows. In the initial state () depicted in Figure 0(a), the robots execute the actions and resulting in a new state () depicted in Figure 0(b). However, the NM only observes the action of robot , because this action takes place in a monitored corridor; i.e., . In the next state , the robots execute actions and resulting in a new state () depicted in Figure 0(c). In this case the NM observes two actions; i.e., . Considering these two sets of observed actions the NM is able to infer the dynamic propositions that are known in as follows:
If the NM uses the information about the states and the observed actions, then no violation of the norm is detected and no robot is sanctioned. However, and have violated the norm, since they have moved into an occupied office through non-monitored corridors.
Iii-B Action Reconstruction
NMs use Definition 6 to generate partial state descriptions based on the observed actions. Additionally, we propose that NMs reconstruct the actions that have not been observed. This reconstruction process entails: (i) searching for the actions that have been performed by unobserved agents; and (ii) using the actions found to increase the knowledge about the state of the world. The reconstruction process must be sound, e.g., it cannot indicate that a violation has occurred when it has not in fact occurred. In the following, we introduce full and approximate methods for reconstructing unobserved actions.
Iii-B1 Full Reconstruction
Full reconstruction tries to find exhaustively the actions performed by all the agents that have not been observed. To this aim, the full reconstruction performs a search to identify all solutions to the reconstruction problem.
Given a partial state description corresponding to time (named initial state), a set of observed actions at time , and an partial resulting state corresponding to time (named final state); we define search as a function that computes sets of solutions such that each solution in is a set of actions such that:
the concurrent action is consistent;
the initial state induced by the concurrent action is consistent (i.e., );
the final state induced by the concurrent action is consistent (i.e., ).
Thus, a solution is a set of actions performed by the agents that have not been observed121212If all actions were observed, no reconstruction would be needed. that are consistent with the states of the world before and after the execution of the actions. Given that the NM has a partial knowledge of the states, we do not require that the preconditions (vs. postconditions) of actions in a solution are met in the initial (vs. final) state, since it is possible that the preconditions (vs. postconditions) are true, but the NM is unaware of it.
Example. Given the partial state description , the set of observed actions , and the partial resulting state , the search function looks for actions of agents and (since they are the agents that have not been observed). According to the initial position of , the NM can infer that may have performed two different actions and —these two actions are the only ones consistent with . Similarly, the NM can infer that may have performed three different actions , and —these three actions are the only ones consistent with . However, the actions and are not consistent with the final state —recall that these two actions have as postcondition the fact that is in offices and , respectively; that defines that is in office ; and that defines as inconsistent states where any robot is at two different locations. As a result, the solution set for this problem is defined as:
Once all solutions are found, the NM uses this information to extend the information about the actions performed by unobserved agents and the state of the world. To ensure that the reconstruction is sound, the NM calculates the intersection of actions in the solutions to select actions it is completely sure about (i.e., actions belonging to all solutions). Given a set of search solutions for some initial and final states, we define the reconstruction action set as follows:
If , then the NM expands its knowledge about the actions performed by agents and it uses this information to increase the knowledge about the initial and final states. More formally, the set of actions observed in is updated as:
The initial state is updated as follows:
Finally, the final state is updated as follows:
where is defined as before and is the set of extended invariant literals; i.e., literals in that have not been modified since there is not a solution such that the concurrent action changes any of these literals:
Example. The reconstruction set for the example is:
This action belongs to all solutions, so the NM can be absolutely sure about the performance of this action, even when the NM has not observed it. As a consequence, the NM extends its information as follows:
and remains unchanged and is updated as follows:
The main disadvantage of full reconstruction is that, for many real-world problems, the number of candidate solutions that needs to be explored is prohibitively large, as shown later in Section IV. In response to this problem, we provide a polynomial approximation below.
Iii-B2 Approximate Reconstruction
Approximate reconstruction includes an approximate search that finds the actions performed by unobserved agents that are consistent with the states of the world before and after action execution. Specifically, approximate reconstruction identifies actions that do not necessarily include the specific actions performed by unobserved agents but that allow the NM to control norms. The main intuition beyond approximate reconstruction is as follows: imagine that at a given initial state an agent can perform just one action and that this action is forbidden (vs. mandatory). In this case, the NM identifies that the agent has violated (vs. fulfilled) a norm. Besides that, if an agent can perform different actions and all these actions are forbidden (vs. mandatory), the NM does not need to know which action has been executed to conclude that a norm has been violated (vs. fulfilled)131313Note that the propose of this paper is to monitor norms, not to determine whether agents are responsible for norm violations/fulfilments. Monitoring situations where agents can only execute forbidden/obligatory actions can help to detect norm-design problems. Additionally, the fact that an agent can only execute forbidden actions may be explained by the agent putting itself into these illegal situations (e.g., I am allowed to overtake but overtaking may put me in a situation where I can only exceed the speed limit). . Hence, we say that a violation has been discovered (instead of identified). Given a set of prohibition instances and an action , we define that the action is forbidden (denoted by ) when . Similarly, given a set of obligation instances and an action , we define that the action is mandatory (denoted by ) when .
Given a partial state , a set of observed actions at time , and a partial resulting state ; we define approximate search as a function that calculates the set of all unobserved applicable actions such that:
the preconditions of each action in are consistent with the initial state (i.e., );
the postconditions of each action in are consistent with the final state (i.e., );
actions in are performed by unobserved agents (i.e., );
all unobserved agents perform at least one action in .
Example. Given the partial state description , the set of observed actions , and the partial resulting state , the approximate search function looks for actions of agents and (since they are the agents that have not been observed). According to the initial position of , the NM can infer that may have performed two different actions and . Again, may have performed action . The approximate solution for this problem is defined as:
As in full reconstruction, the NM uses approximate search solutions () to expand its knowledge about the actions performed by unobserved agents and to increase the knowledge about the initial and final states. When an unobserved agent may have executed only one action, then the NM knows for sure that this action was executed. More formally, the reconstruction action set is defined as follows:
The set of actions observed in is updated as:
Then the initial state is updated as follows:
The final state is updated as follows:
where is defined as before and is the set of extended invariant literals in ; i.e., literals that have not been modified since there is not an observed action or an applicable action that changes them:
Finally, the set of discovered violations and fulfilments is a set of actions defined as follows:
where for each action in : is in and the agent that performs (i.e., ) is:
able to execute more than one action (i.e., );
only able to execute forbidden (vs. mandatory) actions and is one of these forbidden (vs. mandatory) actions;
When an agent is only able to perform forbidden (vs. mandatory) actions, an action among these can be selected according to various criteria. For example, in a normative system where the presumption of innocence principle holds, the NM should assume that the agent has violated (vs. fulfilled) the least (vs. most) important norm and the action that violates (vs. fulfils) this norm is selected. Note discovering violations is very useful in many practical applications, in which it would allow the NM to ban offender agents (e.g., Intrusion Detection/Prevention Systems ), to stop the execution of any offender agent (e.g., Business Process Compliance monitoring ), or to put offender agents under close surveillance (e.g., Model-Based Diagnosis Systems ), even when the specific action performed ins not known.
Example. In case of the approximate reconstruction, is only able to perform one action, which entails that the NM can be absolute sure about the performance of this action and the reconstruction set is defined as:
As a consequence, the NM extends its information as follows:
remains unchanged and is updated as follows:
In this situation, is only able to execute forbidden actions —recall that the instances and forbid any robot to move into offices and and that may have been executed actions and . Thus, the set of discovered violations and fulfilments is defined as follows:
note that the discovered violation does not correspond to the action executed by , however, it allows the NM to determine that must have violated an instance.
Iii-C Norm Monitoring
Once all the information about the actions performed by the agents and the partial states has been reconstructed, the NM checks the actions of agents to determine which instances have been violated or fulfilled. Recall that norms in our model are defined as conditional rules that state which actions are obligatory or forbidden. Given that the NM has partial knowledge about the state of the world, the NM should control norms only when it is completely sure that the norms are relevant to ensure that the norm monitoring process is sound. In particular, we define that a norm is relevant to a partial situation when the norm condition is satisfied by the partial situation —i.e., a norm is relevant to a partial situation represented by a partial state , the static properties and the domain knowledge if such that .
Example. In state the norm that forbids robots to move into occupied offices is instantiated three times as follows:
Once the NM has determined which norm instances hold in a given situation, it has to check the actions of agents to determine which instances have been violated and which ones have been fulfilled.
Obligation Instance. In presence of partial knowledge about the actions performed by agents, the NM can only determine that an obligation instance has been fulfilled. If the NM knows all the actions performed by agents, then it can determine whether an obligation has been fulfilled or violated.
Given an obligation instance and a set of observed actions , then the obligation is defined as:
Prohibition Instance. In presence of partial knowledge about the actions performed by agents, the NM can only determine that a prohibition instance has been violated. If the NM knows all the actions performed by agents then it can determine whether a prohibition has been fulfilled or violated.
Given a prohibition instance and a set of observed actions , then the prohibition is defined as:
Finally, the set of discovered violations and fulfilments is used to identify those agents that have violated or fulfilled an instance.
Example. Taking into account the set of actions , the NM can identify that robot has violated the instance , even though this forbidden action has not been observed by the NM. Specifically, there is such that . Besides that, the approximate reconstruction discovers that robot has violated a prohibition instance though it doe snot know the exact action performed—recall that . Had the NM not performed the proposed reconstruction processes, none of these violations would have been detected.
Iv NM Algorithms
Algorithm 1 contains the NM pseudocode. In each step, the NM observes the actions of agents and uses this information to update the current and the previous partial states (lines 4-9). If all the actions have not been observed in the previous state, then the NM executes the function to reconstruct unobserved actions (lines 11-14). Then, the function is executed to determine which norms have been violated and fulfilled in the previous state (line 15) according to Definitions 9 and 10.
Note that the NM code can be executed while actions are performed without delaying agents. Regarding the temporal cost of the algorithm executed by NMs, it is determined by the cost of the function, the implementations of which (full and approximate) are discussed below.
Full Reconstruction (Algorithm 2). This pseudocode corresponds to the full reconstruction function. This function calls the function to search the actions of target agents (line 2). Then, for all the solutions found, the NM checks if they are consistent according to Definition 7 (lines 4-6). Finally, consistent solutions are used to extend the set of observed actions and the knowledge about the initial and final states (lines 7-14). The temporal cost of this algorithm is given by the cost of the function discussed below.
Algorithm 3 contains the pseudocode of the recursive function that computes all the sequences of consistent actions that may have been executed by the agents that have not been observed. It starts by checking that there is at least one target agent (line 2). If so, it identifies all actions that might have been executed by one target agent (lines 3-4). An action might have been executed if it is consistent according to the static properties, the domain knowledge, and the initial and final states. For each consistent action, it reconstructs the actions of the remaining agents recursively (lines 5-13). In the worst case, the temporal cost of this function is , where is the set of agents, is the set of action descriptions and is the maximum number of instantiations per action. This situation arises when no action is observed and all actions are applicable for all agents.
Approximate Reconstruction Function (Algorithm 4). This function calls the function to search the applicable actions per each target agent (line 2). Then, the list of applicable actions per each agent is checked (lines 3-12). Specifically, if an agent may have executed one action only, then the NM knows that this action was executed and it updates the reconstructed action set (lines 3-5). Then, the set of observed actions and the knowledge about the initial and final states is updated (lines 6-12). Finally, discovered violations and fulfilments are calculated (lines 14-19). The temporal cost of this algorithm is given by the cost of the function discussed below.
Algorithm 5 contains the pseudocode of the function. It starts by initialising the list of applicable actions per agent (lines 3-4). Then it calculates the set of instances that are relevant to the initial state (line 7). The function calculates per each target agent the list of applicable actions that it may have executed (lines 9-11). Then, the list of applicable actions per each agent is checked (lines 12-18). Specifically, if an agent may have executed one action only, then the NM knows that this action was executed and it updates the list of applicable actions, the initial and final states, and retracts the agent from the target agents (lines 14-17). This process is repeated until there are no more target agents or the initial and final states remain unchanged. Then the set of instances that are relevant to the initial state is calculated (line 13). Finally, the list of applicable actions per agent is updated with actions of remaining target agents (lines 19-20). The temporal cost of this function is .
This section compares the performance of a NM with full reconstruction, a NM with approximate reconstruction and a traditional norm monitor —which is the method used in the majority of previous proposals [6, 23, 14, 24, 8]— that only considers the observed actions to detect violations; with respect to their capabilities to monitor norm compliance. We have evaluated our proposal in a case study, which allows us to contextualise the results and to give a meaningful interpretation to them; and in a series of random experiments, which allow us to evaluate our proposal under a wide range of different situations and parameter values.
V-a Case Study
We implemented in Java a simulator of the paper example in which robots attend requests in offices connected through corridors. Compliance with the collision avoidance norm is controlled by a monitor that observes surveillance cameras. In each simulation, we generate corridors and cameras randomly. In each step of the simulation, each robot chooses randomly one applicable action to be executed. The simulation is executed steps and repeated times to support the findings. We conducted experiments in which the number of offices