# An Action Language for Multi-Agent Domains: Foundations

## Authors

• 39 publications
• 1 publication
• 11 publications
• 9 publications
• ### Iterated Belief Change Due to Actions and Observations

In action domains where agents may have erroneous beliefs, reasoning abo...
01/16/2014 ∙ by Aaron Hunter, et al. ∙ 0

• ### Dynamic Awareness

We investigate how to model the beliefs of an agent who becomes more awa...
07/06/2020 ∙ by Joseph Y. Halpern, et al. ∙ 0

• ### Relating Knowledge and Coordinated Action: The Knowledge of Preconditions Principle

The Knowledge of Preconditions principle (KoP) is proposed as a widely a...
06/24/2016 ∙ by Yoram Moses, et al. ∙ 0

• ### A Logic of Agent Organizations

Organization concepts and models are increasingly being adopted for the ...
04/28/2018 ∙ by Virginia Dignum, et al. ∙ 0

• ### Probably Approximately Knowing

Whereas deterministic protocols are typically guaranteed to obtain parti...
07/06/2020 ∙ by Nitzan Zamir, et al. ∙ 0

• ### Reasoning about Hypothetical Agent Behaviours and their Parameters

Agents can achieve effective interaction with previously unknown other a...
06/26/2019 ∙ by Stefano V. Albrecht, et al. ∙ 0

• ### Action and Perception as Divergence Minimization

We introduce a unified objective for action and perception of intelligen...
09/03/2020 ∙ by Danijar Hafner, et al. ∙ 10

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction and Motivation

Reasoning about actions and change has been a research focus since the early days of artificial intelligence

[1]; languages for representing actions and their effects were proposed soon after [2]. Although the early papers on this [2] did not include formal semantics, papers with formal semantics came some years later [3]. The approach adopted in this paper is predominantly influenced by the methodology for representing and reasoning about actions and change (RAC) proposed by Gelfond and Lifschitz [4]. In this approach, actions of agents are described in a high-level language, with an English-like syntax and a transition function based semantics. Among other things, action languages provide a succinct way for representing dynamic domains. The approach proposed in this paper is also related to action description languages developed for planning, such as [5, 6].

Over the years, several action languages (e.g., , , and ) have been developed [7]. Each of these languages addresses some important problems in RAC (e.g., the ramification problem, concurrency, actions with duration, knowledge of agents). Action languages have also provided the foundations for several successful approaches to automated planning; for example, the language is used in the planner C-Plan [8] and the language is used in CpA [9]. However, the main focus of these research efforts has been about reasoning within single agent domains.

###### Example 1 (Three Agents and the Coin Box)

Three agents, , , and , are in a room. In the middle of the room there is a box containing a coin. It is common knowledge that:

• None of the agents knows whether the coin lies heads or tails up;

• The box is locked and one needs to have a key to open it; only agent has the key of the box;

• In order to learn whether the coin lies heads or tails up, an agent can peek into the box. This, however, requires the box to be open;

• If an agent is looking at the box and someone peeks into the box, then the first agent will notice this fact and will be able to conclude that the second agent knows the status of the coin; yet, the first agent’s knowledge about which face of the coin is up does not change;

• Distracting an agent causes that agent to not look at the box;

• Signaling an agent to look at the box causes this agent to look at the box;

• Announcing that the coin lies heads or tails up will make this a common knowledge among the agents that are listening.

Suppose that the agent would like to know whether the coin lies heads or tails up. She would also like to let the agent know that she knows this fact. However, she would like to keep this information secret from . (Note that the last two sentences express goals that are about agents’ knowledge about other agents’ knowledge.) Intuitively, she could achieve her goal by:

1. Distracting from looking at the box;

2. Signaling to look at the box;

3. Opening the box; and

4. Peeking into the box.

This simple story presents a number of challenges for research in representing and reasoning about actions and their effects in multi-agent domains. In particular:

• The domain contains several types of actions:

• Actions that allow the agents to change the state of the world (e.g., opening the box);

• Actions that change the knowledge of the agents (e.g, peeking into the box, announcing head/tail);

• Actions that manipulate the beliefs of other agents (e.g., peeking while other agents are looking); and

• Actions that change the observability of agents with respect to awareness about future actions (e.g., distract and signal actions before peeking into the box).

We observe that the third and fourth types of actions are not considered in single agent systems.

• The reasoning process that helps to realize that steps (1)-(4) will achieve her goal requires ’s ability to reason about the effects of her actions on several entities:

• The state of the world—e.g., opening the box causes the box to become open;

• The agents’ awareness of the environment and of other agents’ actions—e.g., distracting or signaling an agent causes this agent not to look or to look at the box; and

• The knowledge of other agents about her own knowledge—e.g., someone following her actions would know what she knows.

While the first requirement is the same as for an agent in single agent domains, the last two are specific to multi-agent domains.

With respect to planning, the above specifics of multi-agent systems raise an interesting question:

“How can one generate a plan for the agent to achieve her goal, given the description in Example 1?”

To the best of our knowledge, except a dynamic epistemic modeling system called DEMO [19], there exists no automated planner that can address the complete range of issues as exemplified in Example 1. This is in stark contrast to the landscape of automated planning for single agent domains, where we can find several automated planners capable of generating plans consisting of hundreds of actions within seconds—especially building on recent advances in search-based planning.

Among the main reasons for the lack of planning systems capable of dealing with the issues like those shown in Example 1 are: (i) lack of action-based formalisms that can address the above mentioned issues and that can actually be orchestrated, and (ii) the fact that logical approaches to reasoning about knowledge of agents in multi-agent domains are mostly model-theoretical, and not as amenable to an implementation in search-based planning systems. This is not the case for single-agent domains, where the de-facto standard Planning Domain Description Language (PDDL)

provides a transition-based semantics which allows the creation of highly efficient heuristic search based planners.

In terms of related work, multi-agent actions have been explored in Dynamic Epistemic Logics (DEL) (e.g., [20, 21, 22, 23, 24]). However, as discussed later in the paper, DEL does not offer an intuitive view of how to orchestrate or execute a single multi-agent action. In addition, the complex representation of multi-agent actions (akin to Kripke structures) drastically increases the number of possible multi-agent actions—thus, making it difficult to search for multi-agent action sequences. The research in DEL has also not addressed some critical aspects of multi-agent search-based planning, such as the determination of the initial state of a planning domain instance. Moreover, research in DEL did not explore the link between the state of the world and the observability encoded in multi-agent actions, and hence preventing the dynamic evolution of the observational capabilities and awareness of agents with respect to future actions. In some ways, the DEL approach is similar to the formulations of belief updates (e.g., [25, 26, 27]), and most of the differences and similarities between belief updates and reasoning about actions carry over to the differences and similarities between DEL and our formulation of RAC in multi-agent domains. We will elaborate on these differences in a later section of the paper.

### 1.1 Contributions and Assumptions

Our goal in this paper is to develop a framework that allows reasoning about actions and their effects in a multi-agent domain; the framework is expected to address the above-mentioned issues, e.g., actions’ capability to modify agents’ knowledge about other agents’ knowledge. To this end, we propose a high-level action language for representing and reasoning about actions in multi-agent domains. The language provides an initial set of elements of a planning domain description language for multi-agent systems. The main contributions of the paper are:

• The action language , which considers different types of actions—such as world-altering actions, announcement actions, and sensing actions—for formalizing multi-agent domains, along with actions that affect the dynamic awareness and observation capabilities of the agents;

• A transition function based semantics for , that enables hypothetical reasoning and planning in multi-agent domains;

• The notion of definite action theories, that characterizes action theories for which the computation of the initial state is possible—which is an essential component in the implementation of a heuristic search-based planner for domains described in ; and

• Several theorems relating the semantics of to multi-agent actions characterizations using update models in DEL of [20].

In developing , we make several assumptions and design decisions. The key decision is that actions in our formalism can be effectively executed.222This is not the case in DEL [24], where actions are complex graph structures, similar to Kripke structures, possibly representing a multi-modal formula, and it is not clear if and how such actions can be executed. We also assume that actions are deterministic. This assumption can be lifted in a relatively simple manner by generalizing the treatment of non-deterministic actions, state constraints, and parallel actions studied in the context of single-agent domains.

Next, although we have mentioned both knowledge and beliefs, in this paper we will follow [24, 20] and focus only on formalizing the changes of beliefs of agents after the execution of actions. Following the considerations in [22], the epistemic operators used in this paper can be read as “to the best my information.” Note that, in a multi-agent system, there may be a need to distinguish between knowledge and beliefs of an agent about the world. For example, let us consider Example 1 and let us denote with the proposition “nobody knows whether the coin lies heads or tails up.” Initially, all of the three agents know that is true. However, after agent executes the sequence of actions (1)-(4), will know that is false. Furthermore, also knows that is false, thanks to her awareness of ’s execution of the actions of opening the box and looking into it. However, —being unaware of the execution of the actions performed by —will still believe that is true. If this were considered as a part of ’s knowledge, then would result in having false knowledge.

The investigation of the discrepancy between knowledge and beliefs has been an intense research topic in dynamic epistemic logic and in reasoning about knowledge, which has lead to the development of several modal logics (e.g., [28, 24]). Since our main aim is the development of an action language for hypothetical reasoning and planning, we will be concerned mainly with beliefs of agents. A consequence of this choice is that we will not consider situations where sensing actions, or other knowledge acquisition actions, are employed to correct a wrong belief of agents. For example, an agent will not be able to use a sensing action to realize that her beliefs about a certain property of the world is wrong; or an agent will not be able to use an announcement action to correct an incorrect belief of other agents. Some preliminary steps in this direction have been explored in the context of the DEL framework [21, 29]. We leave the development of an action based formalism that takes into consideration the differences between beliefs and knowledge as future work.

### 1.2 Paper Organization

The rest of the paper is organized as follows. Section 2 reviews the basics definitions and notation of a modal logic with belief operators and the update model based approach to reasoning about actions in multi-agent domains. This section also describes a set of operators on Kripke structures, that will be used in defining the semantics of . Section 3 presents the syntax of . Section 4 defines the transition function of which maps pairs of actions and states into states; the section also presents the entailment relation between action theories and queries. Section 5 explores the modeling of the semantics of using the update models approach. Section 6 identifies a class of theories whose initial states can be finitely characterized and effectively computed. Section 7 provides an analysis of with respect to the existing literature, including a comparison with DEL. Section 8 provide some concluding remarks and directions for future work.

For simplicity of presentation, the proofs of the main lemmas, propositions, and theorems are placed in the appendix.

## 2 Preliminaries

We begin with a review of the basic notions from the literature on formalizing knowledge and reasoning about effects of actions in multi-agent systems. Subsection 2.1 presents the notion of Kripke structures. Subsection 2.2 reviews the notion of update models developed by the dynamic epistemic logic community for reasoning about effects of actions in multi-agent systems. In Subsection 2.3, we investigate a collection of basic operators for modifying Kripke structures, that will be used in our semantic formalization in the following sections.

### 2.1 Belief Formulae and Kripke Structures

Let us consider an environment with a set of agents. The real state of the world (or real state, for brevity) is described by a set of propositional variables, called fluents. We are concerned with the beliefs of agents about the environment and about the beliefs of other agents. For this purpose, we adapt the logic of knowledge and the notations used in [28, 24]. We associate to each agent a modal operator and represent the beliefs of an agent as belief formulae in a logic extended with these operators. Formally, we define belief formulae as follows:

• Fluent formulae: a fluent formula is a propositional formula built using the propositional variables in and the traditional propositional operators , , , , etc. In particular, a fluent atom is a formula containing just an element , while a fluent literal is either a fluent atom or its negation . We will use and to denote true and false, respectively.

• Belief formulae: a belief formula is a formula in one of the following forms:

• a fluent formula;

• a formula of the form where is a belief formula;

• a formula of the form , , , or where are belief formulae;

• a formula of the form or , where is a formula and .

Formulae of the form and are referred to as group formulae. Whenever , we simply write and to denote and , respectively. Let us denote with the language of the belief formulae over and .

Intuitively, belief formulae are used to describe the beliefs of one agent concerning the state of the world as well as about the beliefs of other agents. For example, the formula expresses the fact that “Agent 1 believes that agent 2 believes that is true,” while states that “Agent 1 believes that is true.”

In what follows, we will simply talk about “formulae” instead of “belief formulae,” whenever there is no risk of confusion. The notion of a Kripke structure is defined next.

###### Definition 1 (Kripke Structure)

A Kripke structure is a tuple , where

• is a set of worlds,

• is a function that associates an interpretation of to each element of , and

• For , is a binary relation over .

A pointed Kripke structure is a pair where is a Kripke structure and . In a pointed Kripke structure , we refer to as the real (or actual) world.

Intuitively, a Kripke structure describes the possible worlds envisioned by the agents—and the presence of multiple worlds identifies uncertainty and presence of different beliefs. The relation denotes that the belief of agent about the real state of the world is insufficient for her to distinguish between the world described by and the one described by . The world in the state identifies the world in that corresponds to the actual world.

We will often view a Kripke structure as a directed labeled graph, whose set of nodes is and whose set of edges contains 333 denotes the edge from node to node , labeled by . if and only if . is referred to as an edge coming out of (resp. into) the world (resp. ).

For the sake of readability, we use , , and to denote the components , , and of , respectively. We write to denote the interpretation associated to via and to denote the truth value of a fluent formula with respect to the interpretation .

Following [24], we will refer to a pointed Kripke structure as a state and often use these two terms interchangeably.

In keeping with the tradition of action languages, we will often refer to as the set of fluent literals defined by

 {f∣f∈F,M[π](u)(f)=⊤}∪{¬f∣f∈F,M[π](u)(f)=⊥}.

Given a consistent and complete set of literals , i.e., for every , we write or to indicate that the interpretation is defined in such a way that .

The satisfaction relation between belief formulae and a state is defined as next.

###### Definition 2

Given a formula , a Kripke structure , and a world :

• if is a fluent formula and ;

• if for each such that , ;

• if ;

• if or ;

• if and .

• if for every .

• if for every , where and .

For a Kripke structure and a formula , denotes the fact that for each , while denotes the fact that for every Kripke structure .

###### Example 2 (State)

Let us consider a simplified version of Example 1 in which the agents are concerned only with the status of the coin. The three agents do not know whether the coin has ‘heads’ or ‘tails’ up and this is a common belief. Let us assume that the coin is heads up. The beliefs of the agents about the world and about the beliefs of other agents can be captured by the state of Figure 1.

In the figure, a circle represents a world. The name and interpretation of the world are written in the circle. Labeled edges between worlds denote the belief relations of the structure. A double circle identifies the real world.

We will occasionally be interested in Kripke structures that satisfy certain conditions. In particular, given a Kripke structure we identify the following properties:

• K: for each agent and formulae , we have that .

• T: for each agent and formula , we have that .

• 4: for each agent and formula , we have that .

• 5: for each agent and formula , we have that .

• D: for each agent we have that .

A Kripke structure is said to be a T-Kripke (4-Kripke, K-Kripke, 5-Krikpe, D-Kripke, respectively) structure if it satisfies property T (4, K, 5, D, respectively). A Kripke structure is said to be a S5 structure if it satisfies the properties K, T, 4, and 5. Consistency of a set of formulae is defined next.

###### Definition 3

A set of belief formulae is said to be p-satisfiable (or p-consistent) for if there exists a -Kripke structure and a world such that for every . In this case, is referred to as a -model of X.

Finally, let us introduce a notion of equivalence between states.

###### Definition 4

A state is equivalent to a state if iff for every formula .

### 2.2 Update Models

Update models are used to describe transformations of (pointed) Kripke structures according to a predetermined transformation pattern. An update model uses structures similar to pointed Kripke structures and they describe the effects of a transformation on states using an update operator [20, 23].

Let us start with some preliminary definitions. An -substitution is a set where each is a distinct fluent in and each . We will implicitly assume that for each , the substitution contains . denotes the set of all -substitutions.

###### Definition 5 (Update Model)

Given a set of agents, an update model is a tuple where

• is a set, whose elements are called events;

• each is a binary relation on ;

• is a function mapping each event to a formula in ; and

• is a function mapping each event to a substitution in .

An update instance is a pair where is an update model and , referred to as a designated event, is a member of .

Intuitively, an update model represents different views of an action occurrence which are associated with the observability of agents. Each view is represented by an event in . The designated event is the one that agents who are aware of the action occurrence will observe. The relation describes agent ’s uncertainty on action execution—i.e., if and event is performed, then agent may believe that event is executed instead. defines the action precondition and specifies the changes of fluent values after the execution of an action.

###### Definition 6 (Updates by an Update Model)

Let be a Kripke structure and be an update model. The update operator induced by defines a Kripke structure , where:

• ;

• iff , and ;

• For all , iff and .

The structure is obtained from the component-wise cross-product of the old structure and the update model , by (i) removing pairs such that does not satisfy the action precondition (checking for satisfaction of action’s precondition), and (ii) removing links of the form from the cross product of and if or (ensuring that each agent’s accessibility relation is updated according to the update model).

An update template is a pair where is an update model with the set of events and . The update of a state given an update template is a set of states, denoted by , where

 (M,s)⊗(Σ,Γ)={(M⊗Σ,(s,τ))∣τ∈Γ,(M,s)⊨pre(τ)}
###### Remark 1

In the following, we will often represent an update instance by a graph with rectangles representing events, double rectangles designated events, and labeled links between rectangles representing the relation of agents as in the graphical representation of a Kripke model.

### 2.3 Kripke Structures Operators

The execution of an action in a multi-agent environment will change the real state of the world and/or the beliefs of agents. As we will see later, we will employ the notion of a pointed Kripke structure in representing the real state of the world and the beliefs of agents. As such, the execution of an action in a pointed Kripke structure will modify it. Such a change can be a combination of the following basic changes:444We will focus on viewing a (pointed) Kripke structure as a labeled graph.

• Removing some worlds and/or edges from the structure;

• Adding some worlds and/or edges to the structure; or

• Replicating a structure or merging different structures into one.

In this subsection, we describe some basic operators on Kripke structures that will satisfy the above needs. We will assume that a Kripke structure is given. The first operator on Kripke structures is to remove a set of worlds from .

###### Definition 7 (World Subtraction)

Given a set of worlds , is a Kripke structure defined by:

• ;

• for every and ; and

• .

Intuitively, is the Kripke structure obtained from by removing the worlds in and all edges connected to these worlds. The next two operators are used to remove edges from or introduce edges into a given structure.

###### Definition 8 (Edge Subtraction)

For a set of edges in , is a Kripke structure, , defined by:

• ;

• ; and

• for every .

is the Kripke structure obtained from by removing from all the edges in . Given a state and a set of agents , we denote with the state where

 X=⋃i∈AG∖α{(u,i,v)|(u,v)∈M[i]}.

For a set of edges in , is a Kripke structure, , defined by:

• ;

• ; and

• for every .

While removes edges from the structure, is used to add edges to an existing structure. Recall that the nodes/edges in a Kripke structure indicate uncertainty in the agents’ beliefs. As such, these operators are useful for updating the beliefs of the agents in a domain when the beliefs of the agents change.

###### Definition 10 (Equivalence Modulo c)

Given two Kripke structures and , we say that is c-equivalent555Similar to the notion of bisimulation in [20]. to (denoted as ) if there exists a bijective function , such that for every and , we have that:

1. if and only if ; and

2. for every and , if and only if .

The execution of several types of actions leads to the creation of “duplicates” of an original Kripke structure—e.g., to represent how different groups of agents maintain different perspectives after the execution of an action.

###### Definition 11 (Replica)

Given a Kripke structure , a -replica of is a Kripke structure such that is a bijection between and , , and .

Given a state , a -replica of is a state where is a -replica of and .

We say that and are compatible if for every and every , .

For two compatible Kripke structures and , we define as the Kripke structure, , where:

• ;

• for and for ; and

• .

For a pair of Kripke structures and such that , , , and a one-to-one function , we define as the Kripke structure, , where:

• ;

• for and for ; and

• for each , we have that , and for each we have that

 M′[i]=M1[i]∪M2[i]∪{(u,v)∣u∈M2[S],v∈M1[S],(λ(u),v)∈M1[i]}.

Given two states and such that , , , and a one-to-one function :

 (M1,s1)⊎λα(M2,s2)=(M1⊎λαM2,s2).

Intuitively, the operators and allow us to combine different Kripke structures representing the beliefs of different groups of agents, thereby creating a structure representing the beliefs of “all agents.” The operation effectively unions two Kripke structures (that might have already some worlds in common). The also combines two Kripke structures albeit in a more specific way. It relies on a function to create additional edges between the two structures for agents in .

## 3 The language mA+: Syntax

In this paper, we consider multi-agent domains in which the agents are truthful and no false information may be announced or observed. Furthermore, the underlying assumptions guiding the semantics of our language are the rationality principle and the idea that beliefs of an agent are inertial. In other words, agents believe nothing which they are not forced to believe, and the beliefs of an agent remain the same unless something causes them to change.

In this section and the next section, we introduce the language for describing actions and their effects in multi-agent environment. The language builds over a signature , where is a finite set of agent identifiers, is a set of fluents, and is a set of actions. Each action in is an action the agents in the domain are capable of performing.

Similar to any action language developed for single-agent environments, consists of three components which will be used in describing the actions and their effects, the initial state, and the query language (see, e.g., [7]). We will next present each of these components. Before we do so, let us denote the multi-agent domain in Example 1 by . For this domain, we have that . The set of fluents for this domain consists of:

• : the coin lies tails up ( is often used in place of );

• : agent has the key of the box;

• : the box is open; and

• : agent is looking at the box.

The set of actions for consists of:

• : agent opens the box;

• : agent peeks into the box;

• : agent signals agent (to look at the box);

• : agent distracts agent (so that does not look at the box); and

• : agent announces that the coin lies tail up.

where and . We start with the description of actions and their effects.

### 3.1 Actions and effects

We envision three types of actions that an agent can perform: world-altering actions (also known as ontic actions), sensing actions, and announcement actions. Intuitively,

• A world-altering action is used to explicitly modify certain properties of the world—e.g., the agent opens the box in Example 1, or the agent distracts the agent so that does not look at the box (also in Example 1);

• A sensing action is used by an agent to refine its beliefs about the world, by making direct observations—e.g., an agent peeks into the box; the effect of the sensing action is to reduce the amount of uncertainty of the agent;

• An announcement action is used by an agent to affect the beliefs of the agents receiving the communication—we operate under the assumption that agents receiving an announcement always believe what is being announced.

For the sake of simplicity, we assume that each action falls in exactly one of the three categories.666It is easy to relax this condition, but it would make the presentation more tedious.

In general, an action can be executed only under certain conditions, called its executability conditions. For example, the statement “to open the box, an agent must have its key” in Example 1 describes the executability condition of the action of opening a box. The first type of statements in is used to describe the executability conditions of actions and is of the following form:

 \bf executable a\bf if ψ (1)

where and is a belief formula. A statement of type (1) will be referred to as the executability condition of . is referred as the precondition of . For simplicity of the representation, we will assume that each action is associated with exactly one executability condition. When , the statement will be omitted.

For a world-altering action , such as the action of opening the box, we have statements of the following type that express the change that may be caused by such actions:

 a\bf causes ℓ\bf if ψ (2)

where is a fluent literal and is a belief formula. Intuitively, if the real state of the world and of the beliefs match the condition described by , then the real state of the world is affected by the change that makes the literal true after the execution of . When , the part “” will be omitted from (2). We also use

 a\bf causes Φ\bf if ψ

where is a set of fluent literals as a shorthand for the set .

Sensing actions, such as the action of looking into the box, allow agents to learn about the value of a fluent in the real state of the world (e.g., learn whether the coin lies head or tail up). We use statements of the following kind to represent sensing actions:

 a\bf determinesf (3)

where is a fluent. Statements of type (3) encode a sensing action which enables the agent(s) to learn the value of the fluent . is referred to as a sensed fluent of the sensing action .

For actions such as an agent telling another agent that the coin lies heads up, we have statements of the following kind, that express the change that may be caused by such actions:

 a\bf announcesφ (4)

where is a fluent formula. is called an announcement action.

Let us illustrate the use of statements of type (1)-(4) to represent the actions of the domain .

###### Example 3

The actions of domain can be specified by the following statements:

 \bf executable open(X)\bf if has_key(X),BX(has_key)\bf executable peek(X)\bf if opened,looking(X),BX(opened),BX(looking(X))\bf executable shout_tail(X)\bf if BX(tail),tail\bf executable signal(X,Y)\bf if looking(X),¬looking(Y),BX(¬looking(Y)),BX(looking(X))\bf executable distract(X,Y)\bf if looking(X),looking(Y),BX(looking(X)),BX(looking(Y))open(X)\bf causes openedsignal(X,Y)\bf causes looking(Y)distract(X,Y)\bf causes ¬looking(Y)peek(X)\bf determinestailshout_tail(X)\bf announcestail

where and are different agents in . The first five statements encode the executability conditions of the five actions in the domain. The next three statements describe the effects of the three world-altering actions. is an example of a sensing action. Finally, is an example of an announcement action.

### 3.2 Observability: observers, partial observers, and others

One of the key differences between single-agent and multi-agent domains lies in how the execution of an action changes the beliefs of agents. This is because, in multi-agent domains, an agent can be oblivious about the occurrence of an action or unable to observe the effect of an action. For example, watching another agent open the box allows the agent to know that the box is open after the execution of the action; however, the agent would still believe that the box is closed if she is not aware of the action occurrence. On the other hand, watching another agent peeking into the box does not help the observer in learning whether the coin lies heads or tails up; the only thing she would learn is that the agent who is peeking into the box has knowledge of the status of the coin.

needs to have a component for representing the fact that not all the agents may be completely aware of the presence of actions being executed. Depending on the action and the current situation, we can categorize agents in three classes:

• Full observers,

• Partial observers, and

• Oblivious (or others).

This categorization is dynamic: changes in the state of the world may lead to changes to the agent’s category w.r.t. each action. In this paper, we will consider the possible observability of agents for different action types as detailed in Table 1.

The first row indicates that, for a world-altering action, an agent can either be a full observer, i.e., completely aware of the occurrence of that action, or oblivious of the occurrence of the action. In the second case, the observability of the agent is categorized as other. Note that we assume that the observer agents know about each others’ status and they are also aware of the fact that the other agents are oblivious. The oblivious agents have no clue of anything.

For a sensing action, an agent can either be a full observer—i.e., the agent is aware of the occurrence of that action and of its results—a partial observer—i.e., gaining knowledge that the full observers have performed a sensing action but without knowledge of the result of the observation—or oblivious of the occurrence of the action (i.e., other). Once again, we assume that the observer agents know about each others’ status and they also know about the agents partially observing the action and about the agents that are oblivious. The partially observing agents know about each others’ status, and they also know about the observing agents and the agents that are oblivious. The oblivious agents have no clue of anything. The behavior is analogous for the case of announcement actions.

The dynamic nature of the agents observability can be manipulated and this is specified using agent observability statements of the following form:777As discussed earlier, the “” are omitted from the statements.

 X\bf observes a\bf if φ (5) X\bf aware\_of a\bf if ψ (6)

where is the name of an agent, , and and are fluent formulae. Statements of type (5) indicate that is a full observer of the execution of if holds. Statements of type (6) state that is a partial observer of the execution of if holds. , , and (resp. ) are referred to as the agent, the action, and the condition of (5) (resp. (6)).

In the following, we will assume that, for every agent and for every action , if occurs in a statement of the form (5), then it will not occur in a statement of the form (6) such that is consistent and vice versa.

###### Definition 13

An domain is a collection of statements of the forms (1)-(6).

### 3.3 More Examples of mA+ Domains

In this section, we illustrate the use of in representing multi-agent domains. We start by completing the specification of the domain .

###### Example 4 (Observability in D1)

The actions of are described in Example 3. The observability of agents in can be described by the set of statements

 X\bf observes open(X)X\bf observes peek(X)Y\bf observes open(X)\bf if looking(Y)Y\bf aware% \_of peek(X)\bf if looking(Y)Y\bf observes shout_tail(X)X\bf observes shout_tail(X)X\bf observes distract(X,Y)X\bf observes signal(X,Y)Y\bf observes distract(X,Y)Y\bf observes signal(X,Y)

where and denote different agents in . The above statements say that agent is a fully observant agent when , , , , or are executed; is a fully observant agent if it is looking (at the box) when is executed. is a partially observant agent if it is looking when is executed. An agent different from or is oblivious in all cases.

The next example represents a domain in which the agent who executes an action might not be a full observer of the action occurrence.

###### Example 5

Let us consider a domain with two agents . The two agents are operating in a room; agent is blind while is not. Both agents are aware that by flipping a switch it is possible to change the status of the light in the room, and both agents can perform such action. On the other hand, the effect of the execution of the action will be visible only to . This action can be described by the following statements of :

 flip\bf causes on\bf if% ¬onflip\bf causes ¬on\bf if on% B\bf observes flip

We will next describe several examples that contain common actions that are typical to multi-agent domains in . We refer to these actions as reference setting actions, such as the action of distracting another agent. These types of actions are interesting, because it is often necessary for agents to execute them in order to allow subsequent actions to achieve their intended effects (e.g., sharing a secret).

###### Example 6 (Distraction)

Agent wishes to access some files on ’s computer without ’s knowledge. Agent is rather observant therefore, in order for to succeed, will need to first cause a distraction (such as pulling the fire alarm) in order to pull ’s attention away from the computer. Once is distracted, may access the file on the computer. This can be described by the following statements:

 distract(A,C)\bf causes % distracted(C)A\bf observes distract(A,C)C\bf observes distract(A,C)\bf executable accessFile(A,C)\bf if % distracted(C)

The action that helps an agent to form or dissolve a group is also frequently needed in multi-agent domains. Groups enable, for example, the execution of communications that are only local to the group (e.g., a secret conversation).

###### Example 7 (Group Formation/Dissolution and Secret Communication)

Continuing with Example 6, now that has access to the file, she needs the assistance of agent to learn its contents because the file is encrypted and the expertise of is required for decryption. In order to read the file, must first establish a connection with —agent must open/invite a communications channel to . Let denote that and are connected and denote that is distracted. This action of inviting to connect via some communication channel can be represented using the action with the following following specification:

 openChannel(A,B)\bf causes % linked(A,B)A\bf observes openChannel(A,B)B\bf observes openChannel(A,B)C\bf observes openChannel(A,B)\bf if ¬distracted(C)\bf executable openChannel(A,B)\bf if ¬linked(A,B)

Once a channel has been opened, agents and are linked and they may together read the file. Once they have read the file, they disconnect the channel in order to leave no trace of their activity. This action can be represented using the action with the following specification:

 closeChannel(A,B)\bf causes ¬% linked(A,B)A\bf observes closeChannel(A,B)B\bf observes closeChannel(A,B)C\bf observes closeChannel(A,B)\bf if ¬distracted(C)\bf executable closeChannel(A,B)\bf if % linked(A,B)

Reading the file allows and to understand its content. Let us assume that the file indicates whether tomorrow is the date set for a cyber-attack against ’s organization. This can be represented as follows.

If a channel is open, it can be used to share the knowledge of an impeding attack. However, the communication is secure only if the third party is distracted. This action is an announcement action and can be represented using the following statements.

 warnOfAttack(A,B)\bf announces% attackDate(tomorrow)A\bf observes warnOfAttack(A,B)B\bf observes warnOfAttack(A,B)C\bf observes warnOfAttack(A,B)\bf if ¬distracted(C)\bf executable warnOfAttack(A,B)\bf if % linked(A,B)∧attackDate(tomorrow)

A more general version of the actions of secretly creating/dissolving a group is given in the next example.

###### Example 8

Consider an agent joining an agent to gain visibility of everything that the agent does; this can be modeled by the action join, where joins at the same level of visibility of ’s actions:

 join(Y,X)\bf causes group\_% member(Y,group(X))X\bf observes join(Y,X)Y\bf observes join(Y,X)

This could be refined by adding the need to be invited to join :

 \bf executable join(Y,X)\bf if invited(Y,X)

The effect of gaining visibility of the actions of can be described by

 Y\bf observes aX\bf if group\_member(Y,group(X))

where is any action that is executed by agent . The symmetrical operation is the operation of leaving the group, leading the agent to completely loose visibility of what agent is doing:

 leave(Y,X)\bf causes ¬group\_% member(Y,group(X))X\bf observes leave(Y,X)Y\bf observes leave(Y,X)

The agent may also decide to take the action of separating from the group, through the action separate, where the agent will observe from “a distance”, with the consequent loss of the intimate knowledge of actions’ effects:

 separate(Y,X)\bf causes ¬% group\_member(Y,group(X))∧group\_observer(Y,group(X))X\bf observes separate(Y,X)Y\bf observes separate(Y,X)Y\bf aware\_of aX\bf if group\_observer(Y,group(X))

Distracting an agent is not only necessary for secure communication, it is also important for certain world-altering actions to achieve their intended effects, as in Example 6. Another example that highlights the importance of this type of actions can be seen next.

###### Example 9

Agent is a prisoner, having been captured by . Agent is in charge of rescuing agent . In order to do so, he must first distract , and then trigger a release on ’s computer. Once the release has been triggered, may escape. The distract action has already been presented in Example 6. Let us consider the other actions. Rescuing an agent means that the rescued agent is released. We use the following statements:

 rescue(A,D)\bf causes released(D)A\bf observes rescue(A,D)D\bf observes rescue(A,D)C\bf observes rescue(A,D)\bf if ¬% distracted(C)\bf executable rescue(A,D)\bf if % distracted(C)

The action of escaping can have different effects.

 escape(D)\bf causes dead(D)\bf if ¬distracted(C)escape(D)\bf causes free(D)\bf if distracted(C)A\bf observes escape(D)D\bf observes escape(D)C\bf observes escape(D)\bf if ¬% distracted(C)

### 3.4 Initial State

A domain specification encodes the actions and their effects and the observability of agents in each situation. The initial state, that encodes both the initial state of the world and the initial beliefs of the agents, is specified in using initial statements of the following form:

 \bf initially φ (7)

where is a formula. Intuitively, this statement says that is true in the initial state. We will later discuss restrictions on the formula to ensure the computability of the Kripke structures describing the initial state.

###### Example 10 (Representing Initial State of D1)

Let us reconsider Example 1. The initial state of can be expressed by the following statements:

 \bf initially C(has_key(A))\bf initially C(¬has_key(B))\bf initially C(¬has_key(C))\bf initially C(¬opened)\bf initially C(¬BXtail∧¬BX¬tail)for X∈{A,B,C}\bf initially C(looking(X))for X∈{A,B,C}

These statements indicate that everyone knows that has the key and and do not have the key, the box is closed, no one knows whether the coin lies head or tail up, and everyone is looking at the box.

The notion of an action theory in is defined next.

###### Definition 14 (Action Theory)

A -action theory is a pair where is a domain and is a set of statements.

## 4 Transition Function for mA+ Domains

A domain specifies a transition system, whose nodes are “states” that encode the description of the state of the world and of the agents’ beliefs. This transition system will be described by a transition function , which maps pairs of actions and states to states. For simplicity of the presentation, we assume that only one action is executed at each point in time—it is relatively simple to extend it to cases where concurrent actions are present, and this is left as future work. As we have mentioned in Section 2, we will use pointed Kripke structures to represent states in action theories. A pointed Kripke structure encodes three components:

• The actual world;

• The state of beliefs of each agent about the real state of the world; and

• The state of beliefs of each agent about the beliefs of other agents.

These components are affected by the execution of actions. Observe that the notion of a state in action theories is more complex than the notion of state used in single-agent domains (i.e., a complete set of fluent literals).

Let us consider a state . We say that an action is executable in if the executability condition of , given by a statement of the type (1) in , is satisfied by , i.e., . The effect of executing an action in is, in general, a set of possible states denoted by . Since each type of action will impact in a different manner, we define

 ΦD(a,(M,s))=⎧⎪ ⎪⎨⎪ ⎪⎩ΦwD(a,(M,s))ifa is a world-altering actionΦsD(a,(M,s))ifa is a sensing actionΦaD(a,(M,s))