Argumentation has recently become one of the main approaches for non-monotonic reasoning and multi-agent interaction in artificial intelligence and computer science[4, 7, 41]
. The most prominent approach in argumentation models is probably the abstract argumentation framework (AAF) by Dung. In AAF, the contents of the arguments are abstracted from and the framework can be represented as a directed graph in which nodes represent arguments (a set ), and arcs between these nodes represent binary defeat relations (denoted as ) over them.
An important question is which arguments to accept. In his seminal paper, Dung has defined extension-based semantics which correspond to different criteria of acceptability of arguments. For example, if we have two arguments that defeat each other, we cannot accept both. We may accept only one of them. Another equivalent labeling-based semantics is proposed by Caminada [18, 14]. Using this approach, an argument is labeled (i.e. accepted), (i.e. rejected), or (i.e. undecided).
One of the essential properties, that is common, is the condition of admissibility: that accepted arguments must not attack one another, and must defend themselves against counter-arguments, by attacking them back. A stronger notion is called completeness, and is captured, in terms of labelings, in the following two conditions:
An argument is labeled accepted (or in) if and only if all its defeaters are rejected (or out).
An argument is labeled rejected (or out) if and only if at least one of its defeaters is accepted (or in).
In all other cases, an argument should be labeled undecided (or undec). Thus, evaluating a set of arguments amounts to labeling each argument using a labeling function to capture these three possible labels. Any labeling that satisfies the above conditions is a legal labeling, and corresponds to a complete labeling (to be discussed in more detail below). Every complete (i.e. legal) labeling represents a consistent self-defending point of view. We will use legal labeling and complete labeling interchangably.
Since there can be different reasonable positions regarding the evaluation of an argumentation graph, choosing one legal labeling above another is not a trivial task. Therefore, in a multi-agent setting, different agents can subscribe to different positions. Hence, a group of agents with an argumentation graph would need to find a collective labeling that best reflects the opinion of the group. Consider the following example which is depicted in Figure 1.
Example 1 (A Murder Case).
A murder case is under investigation. There is an argument that the suspect is innocent, which suggests that he should be set free (). However, there is some evidence that the suspect was at the crime scene during the crime time, which suggests that the suspect is not innocent (). Weirdly enough, a witness confirmed that she saw someone who looks like the suspect in a bar during the crime time, which suggests that the suspect is innocent ().
Clearly, and defeat each other since they support negating conclusions. Also, defeats since it provides enough evidence to nullify it.
A team of four jurors has been assigned to decide on this case. They have been provided with the previous information. Figure 1 shows the three possible legal labelings. Each juror’s judgment can correspond to only one of these labelings. Suppose they voted as shown in Figure 1 (the four thumbs-ups), what would be a labeling that best reflects the opinion of the team?
Despite the apparent simplicity of the problem, the aggregation of individual evaluations can result into an inconsistent group outcome i.e. even when each individual submits a legal labeling, the aggregation outcome might not be a legal labeling. This problem of aggregating labelings can be compared to preference aggregation (PA) [1, 2, 26, 45], judgment aggregation (JA) [33, 31, 32, 30], and non-binary judgment aggregation [22, 23]. These areas have so far blossomed around impossibility results. There exist many differences between labelings and preference relations stemming from their corresponding order-theoretic characterizations. Labeling aggregation differ from JA in that arguments (which are the counterparts of propositions) can have three values instead of two traditionally considered in JA. Considering the general framework in , our settings can be considered as focusing on special classes of feasible evaluations, which are the conditions imposed by the legal labelling (or other semantics). Additionally, the possible evaluations of each issue (argument, in our case) are to accept (labels as ), reject (labels as ), or be undecided (labels as ). However, translation of results between labeling aggregation and non-binary JA amounts to encoding argument semantics in propositional logic, which is not a trivial task [5, 6].
Recently, the problem of aggregating valid labelings has been the topic of some studies [42, 3, 15, 9, 11]. In [42, 3], the argument-wise plurality rule (AWPR) which chooses the collective label of each argument by plurality, independently from other arguments, was defined and analyzed. On the other hand, Caminada and Pigozzi  showed how judgment aggregation concepts can be applied to formal argumentation in a different way. They proposed three possible operators for aggregating labelings, namely the skeptical operator, the credulous operator, and the super credulous operator. These operators guarantee not only a well-formed outcome but also a compatible one, that is, it does not go against the judgment of any individual.
In order to assess the three operators, we assume that individuals have preferences over the outcomes. Although the outcomes of the three aggregation operators proposed in  are compatible with every individual’s labeling, this does not mean that they are the most desirable given individuals’ preferences. It is possible that other compatible labelings are more desirable. Moreover, it is possible that some agents submit an insincere opinion in order to get more desirable outcomes. Given that, it is interesting to study the following two questions:
Are the social outcomes of the aggregation operators in  Pareto optimal if preferences between different outcomes are also taken into consideration?
How robust are these operators against strategic manipulation? And what are the effects of strategic manipulation from the perspective of social welfare?
The first question studies the Pareto optimality of the outcomes of these operators. A Pareto optimal outcome (given individuals preferences) cannot be replaced with another outcome that is more preferred by all individuals and is strictly more preferred by at least one individual. Pareto optimality is a fundamental concept in any social choice setting and a clearly desirable property for any aggregation operator.
The second question studies the strategy proofness of the operators. Strategy proofness is fundamental in any realistic multi-agent setting. A strategy-proof operator is one that produces outcomes where individuals have no incentive to misrepresent their votes (i.e. to lie). Unfortunately, as we will see later, most strategy proofness results for the three operators are negative. However, we show later that lies do not always have bad effects on other agents.
One can realize that individuals’ preferences (over all the labelings) play a vital role in answering the previous two questions. However, aggregation operators usually do not give the chance for individuals to disclose these preferences. The labeling an agent submits is the only information available about agent’s preferences. It seems a natural choice to assume that the submitted labeling is the most preferred one according to agents’ individual preference. Moreover we assume that the rest of agent’s preferences can be modeled using distance from the most preferred one. For example, if the top preferred outcome for agent is the outcome (i.e. , ), then iff where is the distance between the two outcomes and .
In this work, we investigate different classes of preferences based on different distance measures, and use them to analyze the three aggregation operators proposed in  with respect to the aforementioned two questions.
This paper makes three distinct contributions. First, it introduces the first thorough study of Pareto optimality and strategy proofness for aggregation operators in the context of argumentation. In doing so, the paper highlights that considering argumentation in multi-agent conflict resolution calls for criteria other than logical consistency such as social optimality and strategic manipulation.
Second, the paper introduces different families of agents’ preferences. Building on the new concept of issue, proposed by Booth et al. , we define a new class of agents preferences. We also define a new class of preferences which consider the label as a middle label between and . These new families of preferences capture the intuitions, are more natural, and broaden the scope of analysis of preferences.
The third contribution of this paper is establishing relations between the different classes of preferences. Some of these relations hold for any aggregation operator and others for some special aggregation operators. Additionally, we provide a full comparison for three previously introduced labeling aggregation operators with respect to the proposed classes of preferences. Moreover, we also consider cases where agents do not share the same classes of preference. Our results are based on two fundamental criteria, namely Pareto optimality and strategy proofness. For most classes of preferences we establish the superiority of the skeptical operator. However, we also characterize situations where the credulous and super credulous operators are as good as the skeptical operator. This highlights a trade-off between the two criteria on one hand, and seeking more committed outcomes on the other hand.
Our results bridge a gap in our understanding of the social optimality and strategic manipulation of labeling aggregation operators. As for the Pareto optimality, we show the persistence of the superiority of the skeptical operator. However, there are situations where the credulous and super credulous operators are as good as the skeptical operator. This has an implication on the choice of the appropriate aggregation operator given the criteria that is considered more important, as well as, the nature of agents preferences.
As for the strategy proofness, we establish the fragility of the three operators against strategic manipulation. This negative result is consistent even for a wide range of individual agent preference criteria (except for two cases). This highlights a major limitation of these otherwise attractive approaches to collective argument evaluation.
Despite the negative results, our results show that lies with the skeptical operator are always benevolent i.e. every strategic lie by an agent does not hurt others, but rather improves their welfare. Furthermore, we show that this effect is surprisingly consistent for a wide range of individual agent preference criteria. This shows an important advantage for such an approach to labeling aggregation.111Part of the results of this paper have been presented in .
2.1 Abstract Argumentation Framework (AAF).222Readers familiar with AAF can skip this part.
The seminal paper by Dung  introduced the fundamental notion of abstract argumentation framework that can be represented as a directed graph where the vertices represent arguments (ignoring details about their contents) and the directed arcs represent the defeat relations between these arguments.444We will use “argumentation graph” and “argumentation framework” interchangeably. For example, in Figure 2, argument is defeated by arguments and which are, in turn, defeated by arguments and .
Definition 1 (Argumentation framework ).
An argumentation framework is a pair where is a finite set of arguments and is a defeat relation. We say that an argument defeats an argument if (sometimes written ).
There are two approaches to define semantics that assess the acceptability of arguments. One of them is extension-based semantics by Dung , which produces a set of arguments that are accepted together. Another equivalent labeling-based semantics is proposed by Caminada [18, 14], which gives a labeling for each argument. With argument labelings, we can accept arguments (by labeling them as ), reject arguments (by labeling them as ), and abstain from deciding whether to accept or reject (by labeling them as ). As  employed the labeling approach, so we continue to use it here.
Let be an argumentation framework. An argument labeling is a total function .
For the purposes of this paper, we use the following marking convention, as shown in Figure 3, arguments labeled are shown in white, in black, and in gray.
We write for the set of arguments that are labeled by , for the set of arguments that are labeled by , and for the set of arguments that are labeled by . A labeling can be represented as ,,. Equivalently, we also denote a labeling as: .
However, labelings should follow some given conditions. If an argument is labeled then all of its defeaters are labeled . If an argument is labeled then at least one of its defeaters is labeled . We call a labeling that follows the previous two conditions an admissible labeling.
Let be an argumentation framework. An admissible labeling is a mapping such that for each it holds that:
if then , and
if then .
Some examples for admissible labelings, in Figure 2, can include the following: , , , , and .
One can realize that in an admissible labeling, unlike -labeled and -labeled arguments, -labels do not need to be justified i.e. an argument can be labeled under an admissible labeling without any condition.
The complete semantics, however, force -labels to be also justified. A complete labeling is an admissible labeling with the following extra condition: If an argument is labeled then there is no defeating argument that is labeled (that is, there is insufficient ground to label the argument ) and not all defeating arguments are labeled (that is, there is insufficient ground to label the argument ). We call a labeling which follows these rules a complete labeling.
Let be an argumentation framework. A complete labeling is a mapping such that for each it holds that:
if then ,
if then , and
As an example for a complete labeling, in Figure 2, we have only one complete labeling, namely .
2.2 Aggregation Operators
Perhaps, the most common aggregation rules are the majority rules, in which an alternative is chosen if and only if it receives a number of votes that exceeds some presepecified threshold , where is the number of voters. However, these rules are not always appropriate. One example is in juries, when the legal or the moral responsibility of the outcome is shared by all individuals. Indeed Ronnegard  argued that the attribution of moral responsibility to all members of a committee is legitimate when the decision is taken through unanimous voting, while it is not necessarily the case otherwise. Another example is when the outcome of the decision can potentially harm some individuals. It was shown in  that people show a preference for more conservative aggregation procedures when the outcome of the decision may involve the infliction of personal harm. Aiming to address such specific scenarios, Caminada and Pigozzi  proposed three aggregation rules that ensure the compatibility of the outcome with all individuals votes.
Before introducing the aggregation operators that were defined in , we first define the problem of aggregation. The problem of labeling aggregation can be formulated as a set of individuals that collectively decide how an argumentation framework must be labelled.
Definition 5 (Labeling aggregation problem ).
Let be a finite non-empty set of agents, and be an argumentation framework. A labeling aggregation problem is a pair .
Each individual has a labeling which expresses the evaluation of by this individual. A labeling profile is a set of the labelings submitted by agents in : .555We follow  in assuming that the profile is a set of labelings instead of a list of labelings. Although this is not common in judgment aggregation literature where the number of votes matter in many operators, it is not the case for the three operators considered in this study, since they focus on compatibility instead of cardinality. As such, although we list labelings in the profile, it is possible that a profile has less than elements, since agents can submit similar labelings.
A labeling aggregation operator is a function that maps a set of labelings, chosen from the set of all labelings, , into a collective labeling.666Although it would be more precise to use to denote the set of all labelings for according to semantics , we will often drop and , and use instead when there is no ambiguity about the argumentation framework. The same goes for all other notations (e.g. ) that were defined for an , when there is no ambiguity about the argumentation framework.
Definition 6 (Labeling aggregation operator ).
Let be a labeling aggregation problem. A labeling aggregation operator for is a function such that , where is the collective labeling.
A labeling is said to be less or equally committed than another labeling if and only if every argument that is labeled by is also labeled by and every argument that is labeled by is also labeled by .
Definition 7 (Less or equally committed ).
Let and be two labelings of argumentation framework . We say that is less or equally committed as () iff .
Two labelings and are said to be compatible with each other if and only if for every argument, there is no conflict between the two. In other words, every argument that is labeled by is not labeled by and every argument that is labeled by is not labeled by .
Definition 8 (Compatible labelings ).
Let and be two labelings of argumentation framework . We say that is compatible with () iff
We now define a compatible operator as the following:
Definition 9 (Compatible operator).
Let be a labeling aggregation problem, and let be a labeling aggregation operator for . We say is a compatible operator if given any labeling profile , i.e. the outcome of is compatible with each individual’s labeling.
In , Caminada and Pigozzi proposed three different aggregation operators, namely the skeptical operator, the credulous operator and the super credulous operator. Each of these operators maps a set of labelings, that are submitted by individuals, into a collective labeling. The following two definitions are used in the definition of these operators:
Definition 10 (Initial operators , ).
Let be a labeling aggregation problem. The skeptical initial and credulous initial operators are labeling aggregation operators for defined as the following:
777We will often use and to refer to the skeptical initial and credulous initial operators, respectively.
Definition 11 (Down-admissible and up-complete labelings ).
Let be a labeling of argumentation framework . The down-admissible labeling of , denoted as , is the biggest element of the set of all admissible labelings that are less or equally committed than :
where is the set of all admissible labelings for . The up-complete labeling of , denoted as , is the smallest element of the set of all complete labelings that are bigger or equally committed than .
Now, we provide the definitions of the three operators:
Definition 12 (Skeptical , Credulous and Super Credulous operators ).
Let be a labeling aggregation problem. The skeptical , the credulous and super credulous operators are labeling aggregation operators for defined as the following:
Given the set of all admissible labelings for some argumentation framework, it is shown that the outcome of the skeptical aggregation operator is the biggest element in that is less or equally committed to every individual’s labeling.
Theorem 1 ().
Let () be labelings of argumentation framework . Let be . It holds that is the biggest admissible labeling such that for every .
2.3 Distance Measures
In this part, we define the family of distance measures that we use to define preferences. Each of the distance measures we consider is characterized by three choices:
Individual arguments vs. Issues (set of arguments).
Set inclusion vs. Quantitative distance.
Uniform vs. in the middle.
The combination of all of these choices produces eight different distance measures. We start from the third choice. The uniform vs. in the middle choice captures the intuition that the distance between accepting an argument () and rejecting it () may be set as equal or superior to the distance of accepting (or rejecting) an argument ( or ) and abstaining on the same argument (). In other words, an / disagreement may be as serious or more serious (depending on the contexts) than a / (or a /) disagreement.
Thus, we consider the following two cases. First, , , and are equally distant from each other. In other words, , where is the difference between two labels for one argument, and is either or . In the other case, we assume that is in the middle between and . Thus, we differentiate between two types of disagreement. One between and , and the other between and . When considering distance, we assume .
2.3.1 Case 1: , , and are Equally Distant from Each Other
Hamming Set and Hamming Distance
The Hamming set between two labelings and is the set of arguments that these two labelings disagree upon.
Definition 13 (Hamming Set ).
Let , be two labelings of . We define the Hamming set between these two labelings as:
The Hamming distance between two labelings and is the number of arguments that these two labelings disagree upon.
Definition 14 (Hamming Distance ).
Let and be two labelings of . We define the Hamming distance between these two labelings as:
Issue-wise Set and Issue-wise Distance.
The label of an argument depends on the labels of the defeating arguments. Therefore, measuring the distance by treating arguments independently might not give an accurate sense of how far two labelings are from each other. Consider the example in Figure 4. Using Hamming distance, we have .
However, one can argue that is closer (than ) to . Intuitively speaking, if and further agreed on the labeling of (or ), then they would have been equivalent. On the other hand, and should further agree on (or ) and (or ) in order to become equivalent. In other words, the number of arguments whose labelings need to be switched in order to make the two labelings be equivalent is less between and than between and .
Motivated by this example, Booth et al.  proposed a new distance method, using the notion of “issue”, which they defined. This distance method captures the idea in the previous example, while satisfying a set of axiomatic properties which they listed as essential for any distance measure.
Crucial to the definition of the “issue” is the concept of “in-sync”. We say that two arguments and are in-sync if for any pair of labelings , cannot be changed to without causing a change of equal magnitude when moving from to , and vice versa.
Definition 15 (in-Sync for semantics ).
Let be the set of all labelings according to semantics for argumentation framework . We say that two arguments are in-sync for semantics ():
In-sync is an equivalence relation. We can partition the set of arguments in any argumentation framework into the in-sync equivalence classes, which form what is called issues.888The definition of issue, along with all the definitions depending on it, can be defined for semantics (as the case for “in-sync”). However, from now on, we will restrict all of these definitions to the complete semantics, and drop the letter . Thus, “issues” in what follows refers to the equivalnce classes of in-sync for the complete semantics.
Definition 16 (Issue ).
Given the argumentation framework , a set of arguments is called an issue iff it forms an equivalence class of the relation in-Sync ().
The Issue-wise set between two labelings and is the set of issues that these two labelings disagree upon.
Definition 17 (Issue-wise Set ).
Let , be two labelings of and let be the set of all issues in . We define the Issue-wise set between these two labelings as:
Note that the sentence “for some (equiv. all)” follows from the definition of issues. One can realize that:
The Issue-wise distance between two labelings and is the number of issues that these two labelings disagree upon.
Definition 18 (Issue-wise Distance ).
Let , be two labelings of . We define the Issue-wise distance between these two labelings as:
For example, in Figure 4, the Issue-wise sets between and the other two labellings are:
While the corresponding Issue-wise distances are:
2.3.2 Case 2: is in the Middle between and
In this section, we consider the case where is in the middle between and . Thus, we differentiate between two types of disagreement: 1) disagreement, and 2) disagreement. When considering distance, we assume . 999 The use of here is chosen carefully to satisfy the triangle inequality. However, the use of any s.t. would not affect the results of this paper. We just use here for simplicity.
To illustrate the difference from the previous case, consider the example shown in Figure 5. In this example, one can realize that the labelings and are equally distant from labeling when considering Hamming set/distance or Issue-wise set/distance.
However, one can argue that is closer than to . Consider the arguments in Figure 5. Labelings and seem to be on completely different sides regarding their evaluations for and . On the other hand, the difference between and is less drastic, because abstains from taking any position about and .
We use IUO (short for In-Undec-Out i.e. Undec is in the middle) to denote this class of preferences.
IUO Hamming Sets and IUO Hamming Distance
The Hamming set () between two labelings and is the set of arguments that both labelings label as decided (i.e. or ), but on which they disagree upon. The Hamming set () between two labelings and is the set of arguments that one of the two labelings labels as decided (whether or ) and the other labels as undecided.
Definition 19 (IUO Hamming sets ).
Let , be two labelings of . We define the IUO Hamming sets as a pair , where is Hamming set and is Hamming set:
where is the set of decided ( or ) arguments according to the labeling .
The IUO Hamming distance between two labelings and is the number of arguments in added to twice the number of the arguments in .
Definition 20 (IUO Hamming Distance ).
Let , be two labelings of . We define the IUO Hamming distance between these two labelings as:
IUO Issue-wise Sets and IUO Issue-wise Distance
The Issue-wise set () between two labelings and is the set of issues that both of the two labelings label as decided, but on which they disagree upon. The Issue-wise set () between two labelings and is the set of issues that one of the two labelings labels as decided and the other labels as undecided.
Definition 21 (IUO Issue-wise sets ).
Let , be two labelings of and let be the set of all issues in . We define the IUO Issue-wise sets as , where is the Issue-wise set and is the Issue-wise set:
Note that given the definition of issues, for every labeling , an issue is either decided (all arguments in it are labeled or by ) or undecided (all arguments in it are labeled undecided by ):
The IUO Issue-wise distance between two labelings and is the number of issues in added to twice the number of the issues in .
Definition 22 (IUO Issue-wise Distance ).
Let , be two labelings of . We define the IUO Issue-wise distance between these two labelings as:
For example, in Figure 5, the IUO Issue-wise sets between and the other two labellings are:
While the corresponding IUO Issue-wise distances are:
Table 1 summarizes the distance measures we consider.
|Hamming||Set||Hamming Set||IUO Hamming Sets|
|Distance||Hamming Distance||IUO Hamming Distance|
|Issue-wise||Set||Issue-wise Set||IUO Issue-wise Sets|
|Distance||Issue-wise Distance||IUO Issue-wise Distance|
Given the distance measures defined earlier, we define agents’ preferences. We say an agent’s preferences are -based, if her preferences are calculated using the distance measure (e.g. Hamming distance based preferences). We use to denote a weak preference relation by agent whose preferences are -based i.e. for any pair , denotes that is more or equally preferred than by agent with -based preferences. Further, we use to denote a strict preference relation ( iff ), to denote an incomparability relation ( iff ), and to denote an indifference relation ( iff ).
We define the subset relation over pairs of sets as the following.
Definition 23 (Subset Over Pairs ).
Let be four sets, and Let , be two pairs of sets. We use to denote the subset relation over pairs of subsets:
Given a set measure , an agent , who has -set based preferences (and whose top preference is ), would prefer a labelling over another labeling if and only if the set of arguments in is a subset of (where “subset” here refers to the standard definition of subset as well as the definition of “subset over pairs” defined above). Note that the set based preference yields a partial order over the labelings.101010Although formally, the set-based criteria are not measures but mappings to sets, we will slightly abuse terminology and refer to all criteria (set based and distance based) as set and distance measures for easy reference.
Definition 24 (Set Based Preference ).
We say that agent ’s preferences are -set based w.r.t iff:
where is agent ’s most preferred labeling and . Note that -set based preferences is read Hamming set based preferences when , Issue-wise set based preferences when ,
Given a distance measure , an agent , who has -distance based preferences (and whose top preference is ), would prefer a labelling over another labeling if and only if is less than . Note that the distance based preference yields a total pre-order over the labelings.
We now define the classes of preferences which are based on different distance measures, that we defined earlier.
Definition 25 (Distance Based Preference ).
We say that agent ’s preferences are -distance based w.r.t iff:
where is agent ’s most preferred labeling and . Note that -distance based preferences is read Hamming distance based preferences when , Issue-wise distance based preferences when ,
To illustrate the set and distance based preferences, we use Hamming set and Hamming distance based preferences for their simplicity. Consider the example in Figure 6 with four possible complete labelings. The Hamming sets between and the other three labelings are:
Consequently, the Hamming distance values between and the other three labelings are the cardinality values of the Hamming sets between and the other three labelings.
Assume we have agents with Hamming set based preferences. Hence, an arbitrary agent who prefers the most, would have the following preferences: and (neither nor is a subset of the other). However, if agents have Hamming distance based preferences, an agent who prefers the most, would have the following preferences: .
We can now examine the examples in Figure 4 and 5 in the light of preferences. The example in Figure 4 shows how an agent whose top preference is would have different opinions about other labelings given the different distance measures used. If agent has Hamming distance based preferences, then:
then, her preferences would be , while if she has Issue-wise distance based preferences, then:
then, her preferences would be . Hence, it is interesting to introduce the “Issue-wise” concept to define a new class of preferences.
The example in Figure 5 shows how an agent (whose top preference is ) would have different preferences depending on whether , , and are equally distant, or is in the middle between and . In the former case: