Judgment aggregation studies the problems related to aggregating a finite set of yes-no individual judgments, cast on a collection of logically interrelated issues. Such a finite set of issues forms the agenda. It can be seen as a generalisation of preference aggregation .
Until a few years ago, the judgment aggregation literature had focused considerably more on studying impossibility theorems than on developing and investigating specific aggregation rules.
This field development approach departs from the, admittedly much older, field of voting theory. Nevertheless, several recent and independent papers have started to explore the zoo of concrete judgment aggregation rules, beyond the well known premise-based and conclusion-based rules [8, 31]. While the premise- and conclusion-based rules can only be applied if there exists a prior labelling of the agenda issues as premises and conclusions, the following rules are defined for every agenda: quota-based rules , distance-based rules [30, 25, 14, 9], generalizations of Condorcet-consistent voting rules [28, 27, 21], and rules based on the maximisation of some scoring function [21, 3, 36].
Some of these rules obviously generalize well-known voting rules.
However, a ‘compendium’ of existing judgment aggregation rules really does not exist at the moment, despite the several overview papers, chapters and even books that have been published in recent years
However, a ‘compendium’ of existing judgment aggregation rules really does not exist at the moment, despite the several overview papers, chapters and even books that have been published in recent years[24, 18, 12, 1].
Our aim is threefold. First, as there is so far no compendium of judgment aggregation rules, we give one: we list most of the rules that have been proposed recently, in a structured way. This part of the paper does not give novel results, but serves as a partial survey. Second, we compare in a systematic way these rules in terms of inclusion relationships. Third, we consider a few key properties that generalize properties of voting rules (majority-preservation, unanimity, monotonicity, homogeneity and reinforcement) and identify those of the considered rules that satisfy them.
We follow earlier work in judgment aggregation  in using a constraint-based version of judgment aggregation to represent properties like transitivity of preferences. As it is common in voting theory, we consider irresolute rules (also called ‘correspondences’) rather than functions, that is, a rule outputs a non-empty set of collective judgments.
The outline of the paper is as follows. The general definitions are given in Section 2. In Section 3 we review the rules we study in the paper. Majority preservation is a key property of rules, as it generalizes Condorcet-consistency. We focus on majority-preservation in Section 4 and show which of the rules defined in Section 3 satisfy it. In Section 5 we address inclusion and non-inclusion relationships between our rules. In Section 6 we study the rules from the point of view of unanimity, monotonicity, reinforcement and homogeneity. We summarize our contributions in Section 7.
Let be a standard propositional language consisting of well-formed propositional logical formulas, including (tautology) and (contradiction), together with a standard notion of logical consistency. We denote atomic propositions by etc. and formulas from by etc.
An agenda is a finite set of propositions of the form , where for all , and is neither a tautology nor a contradiction, and is a non-negated formula, (i.e., it is not of the form ).We refer to a pair as an issue. The pre-agenda associated with is . We slightly abuse notation and write instead of for .
An agenda is endowed with a notion of consistency which preserves logical consistency. Formally, comes with a set of (-)consistent judgment sets; a (-)consistent judgment set is logically consistent, but the converse does not necessarily hold. Without loss of generality, the agenda’s consistency notion is defined as logical consistency given some fixed formula: a set of formulas is consistent if is logically consistent, where is some exogenously fixed non-contradictory formula, which we call the integrity constraint. This is also the approach followed in  (albeit in the slightly different framework of binary aggregation, where agenda issues are atomic propositions) and in . A similar use of constraints is also done in belief merging theory [19, 16]. When is not specified, by default it is equal to , in which case the notion of consistency associated with the agenda coincides with standard logical consistency.
A judgment on is either or . A judgment set for is a subset of . is complete if and only if for each , either or . A judgment set for is rational if it is complete and consistent. Let be the set of all rational judgment sets for .
For every consistent , the set of rational extensions of , i.e., , is denoted as .
A -profile, or simply a profile, is a finite sequence of rational individual judgment sets, i.e., for some , where is the judgment set of voter . We slightly abuse notation and write when for some , and we write to denote the number of judgment sets in . We sometimes denote as , where . We write (read ” is a sub-profile of ) if for some .
Given two rational judgment sets and we define the Hamming distance : as the number of issues on which and disagree. We also define the Hamming distance between two profiles and as , and between a judgment set and a profile as .
We define as the number of all voters in whose judgment set contains , i.e., .
Consider the pre-agenda . The corresponding agenda is , equipped with the consistency notion corresponding to . The set of rational judgment sets is
Consider the profile with , and . For instance, .
Most often we will write profiles in a table, as in Table 1, with the pre-agenda elements given in the topmost row and the voters’ judgment sets in the leftmost column. If a judgment set contains (respectively ), then we mark this with a “+” (respectively, “-”) in the corresponding column. The constraint, if explicitly defined, will be given in the table caption.
The majoritarian judgment set associated with profile contains all the elements of the agenda that are supported by a strict majority of judgment sets in , i.e., A profile is majority-consistent when is a consistent subset of .
An (irresolute) judgment aggregation rule maps every profile , defined on every agenda , to a nonempty set of rational judgment sets in . When for all profiles , is a singleton, then is said to be resolute. Like in voting theory, resolute rules can be defined from irresolute ones by coupling them with a tie-breaking mechanism.
The preference pre-agendas associated with a set of alternatives are defined by the set of atomic propositions (when , is not a n atomic proposition, but we write as a shorthand for ) and one of these two consistency notions: transitivity, defined as consistency with
or existence of a nondominated alternative, defined as consistency with
Finally, (respectively, ) is defined by its pre-agenda and the consistency notion corresponding to transitivity (respectively, with the existence of a nondominated alternative).
A preference profile over is a finite sequence of linear orders over , which we denote by . The majority graph
associated with is the directed graph whose vertices are elements of and containing edge if and only if a majority of voters in prefer to ; we denote by the number of votes in that prefer to . A social preference function maps every preference profile to a nonempty set of linear orders over . A social choice function (or voting rule) maps every preference profile to a nonempty subset of . With every judgment aggregation rule we can associate two social preference functions, whether we impose the transitivity constraint or the nondominated alternative constraint. From these two social preference functions we can derive two social choice functions by “collecting” the nondominated elements in each of the output preference relations. Sometimes, especially when is odd, these social preference functions or the social choice functions coincide with well-known voting rules (we show several examples in Section
is odd, these social preference functions or the social choice functions coincide with well-known voting rules (we show several examples in Section3).
3 Judgment aggregation rules
We now define five (overlapping) families of judgment aggregation rules. We use the following running example throughout the paper to illustrate the rules.
Let , and the 17-voter profile of Table 1. Consistency in is logical consistency. As is an inconsistent judgment set, is not majority-consistent.
3.1 Rules based on the majoritarian judgment set
A judgment aggregation rule is based on the majoritarian judgment set when for every two -profiles and such that , we have . These rules can be viewed as the judgment aggregation counterpart s of voting rules based on the pairwise majority graph, also known as C1 rules in Fishburn’s classification (see, e.g., ).
Given a set of formulas , is a maximal consistent subset of if and only if is consistent and there exists no other consistent set such that ; and is a maxcard (for “maximal cardinality”) consistent subset of if and only if is consistent and there exists no other consistent set such that . The set of maximal (respectively, maxcard) consistent subsets of is denoted by (respectively, ).
Definition 1 (Maximal Condorcet and maxcard Condorcet rules)
For every -profile , the maximal Condorcet rule (mc) and the maxcard Condorcet rule (mcc) are defined as follows:
Equivalently, . Clearly, .
For the profile of Example 2, the maximal consistent subsets of are , and ; therefore
At least when is odd, it is easy to identify the voting rules obtained from mc and mcc. We give these results informally and without proof:111The proofs can be found in .
For mcc, the transitivity constraint leads to the social preference function that maps a profile to the set of all its Slater orders, i.e., the set of all linear orders over maximising the number of such that if and only if , and that the corresponding voting rule (for odd) is the Slater rule, which maps a profile to the set of all alternatives that are dominating in some Slater order for . If we choose the constraint, then the corresponding voting rule (for odd) is the Copeland rule, which maps a profile to the set of alternatives maximising the number of outgoing edges from in .
For mc, the transitivity constraint (for odd) leads to the top cycle rule, which maps a preference profile to the (unique) smallest subset of such that for every and , we have .222This result has been independently proven (and stated in a stronger way) in . Finally, the choice of the constraint (for odd) leads to the voting rule that maps a profile to its Condorcet winner if and only if the profile has a Condorcet winner, and to the set of all alternatives otherwise.
3.2 Rules based on the weighted majoritarian set
The weighted majoritarian set associated with a profile is the function which, we recall, maps each agenda issue to the number of judgment sets in that contain it. A judgment aggregation rule is based on the weighted majoritarian set when for every two -profiles and , if for every we have , then . These rules can be viewed as the judgment aggregation counterpart s of voting rules that are based on the weighted pairwise majority graph, also known as C2 rules in Fishburn’s classification . Since can be recovered from , every rule based on the majoritarian judgment set is also based on the weighted majoritarian set.
Definition 2 (Median rule)
For every -profile , the median rule (med) is defined as follows:
This rule appears in many places under different names: Prototype , median rule , maximum weighted agenda rule , simple scoring rule  and distance-based procedure . Variants of this rule have been defined by Konieczny and Pino-Pérez  and Pigozzi . For completeness we give here the equivalent distance-based formulation of med, although we consider more generally the family of distance-based rules in Section 3.4. For every -profile , the distance-based rule is defined as follows:
It is not difficult to establish that coincides with med (see , and Proposition 1 in ). The social preference function obtained from med and the choice of the transitivity constraint is the Kemeny social preference function, and the corresponding voting rule is the Kemeny rule.
Consider the agenda and profile of Example 2. We obtain:
As reaches its maximum value (49) for , we have .
The following rule generalizes the ranked pairs voting rule . It proceeds by considering the elements of the agenda in non-increasing order of and fixing each agenda issue value to the majoritarian value if it does not lead to an inconsistency.
Definition 3 (Ranked agenda rule)
Let . For every -profile , ra consists of those judgment sets for which there exists a permutation of the propositions in such that and is obtained by the following algorithmic procedure:
In plain words, ra assigns iteratively a truth value to each proposition of the agenda, whenever it does not produce an inconsistency with propositions already assigned, following an order compatible with . An equivalent non-procedural definition is the following: for every profile , define by: if there is an such that
for all , implies [ if and only if ], and
Then .333The proof—almost straightforward—can be found in .
Consider the profile of Example 2. The highest value of is reached for , therefore is fixed first. Then comes , which is fixed as well. Then come and , tied. We skip both because they would produce inconsistencies; then is fixed, and finally, and . Thus,
Given an -voter profile and a rational judgment set , define and . Given two rational judgment sets , let if and only if there is a such that and for all , . is the set of all undominated rational judgment sets in with respect to .
For the profile of Example 2: , , , . If and , we have , , and , therefore . (In fact, is the only -undominated rational judgment set.)
It is easy to see that the social preference function (respectively, voting rule) associated with ra and the transitivity constraint is the ranked pairs social preference function (respectively, rule), which informally proceeds by iteratively fixing edges in the majority graph, whenever possible, considering all ordered pairs of alternatives
social preference function (respectively, rule), which informally proceeds by iteratively fixing edges in the majority graph, whenever possible, considering all ordered pairs of alternativesin an order corresponding to non-increasing values of , and outputs the rankings obtained this way (respectively, the dominating elements in these rankings). However, the voting rule associated with ra and the constraint is the maximin rule, that maps a profile to the set of alternatives that maximise . The voting rules associated with leximax are refinements of ranked pairs and maximin.
3.3 Rules based on elementary changes in profiles
The next family of rules we consider contains rules that are based on minimal set of changes on a profile needed to render the profile majority-consistent. This family of judgment rules can be viewed as the judgment aggregation counterpart of voting rules that are rationalisable by some distance with respect to the Condorcet consensus class .
The first rule we consider is called the Young rule for judgment aggregation, by analogy with the Young voting rule, which outputs the candidate minimising the number of voters to remove from the profile so that becomes a weak Condorcet winner . The judgment aggregation generalization consists of removing a minimal number of voters so that the profile becomes majority-consistent, or equivalently, to look for majority-consistent subprofiles of maximum cardinality.
Definition 5 (Young rule)
For every -profile ,
Once again we consider and from Example 2. After noticing that removing three judgment sets from restores majority-consistency, and removing less than three judgment sets does not, we obtain
The voting rule associated with y and the constraint is the Young voting rule .
The next rule we define looks for a minimal number of individual judgment reversals in the profile so that becomes majority-consistent, where a judgment reversal is a change of truth value of one agenda element in one individual judgment set. This rule has been proposed first in Miller and Osherson  under the name full. It bears a resemblance with the Dodgson voting rule, but does not exactly correspond to it when choosing either the or the constraint.
Definition 6 (Minimal profile change rule)
For , the mpc rule is defined as:
3.4 Rules based on (pseudo-)distances
For a given constrained agenda, a pseudo-distance on is a function that maps pairs of judgment sets to non-negative real numbers, and that satisfies, for all , , and if and only if .
Two pseudo-distances we will use are the Hamming distance , defined in Section 2, and the geodesic distance444Our name; no name was given of this distance in . on , defined in  as follows. Given three distinct rational judgment sets , we say that is between and if . Let be the graph whose set of vertices is the set of rational judgment sets and that contains an edge between and if and only if there exists no , , between and . Finally, is defined as the length of the shortest path between and in .
Let be a pseudo-distance on and a commutative, associative and non-decreasing function on . The distance-based judgment aggregation rule associated with and is defined as
3.5 Scoring rules
Dietrich  defines a general class of scoring rules for judgment aggregation. Given a function , the rule is defined as
The med rule (3) is a scoring rule (and also a distance-based rule).
The reversal score function rev  is defined as:
The main motivation for introducing this rule is that the associated voting rule (with the transitivity constraint) is the Borda rule. Dietrich  defines four other scoring rules (entailment scoring, disjoint entailment scoring, minimal entailment scoring, and irreducible entailment scoring), two of which generalize the Borda rule as well. As he focuses on reversal scoring, we do as well, and leave the other four for further study beyond this paper.
Duddy et al.  introduce another interesting and intriguing scoring rule (defined only when the agenda satisfies a specific property); it generalizes not only the Borda rule, but also a well-behaved mean rule for finding collective dichotomies. We leave it for further study as well.
Intuitively, a judgment aggregation rule is majority-preserving if and only if returns only the extensions of the majoritarian judgment set whenever it is consistent. In case of ties, a majoritarian set can have more than one extension. For example, when we have agenda and individual judgments and , then , which can be extended into two complete collective judgment sets, namely and .
A judgment aggregation rule is majority-preserving if and only if for every agenda and for every majority-consistent -profile we have . A rule is weakly majority-preserving if and only if for every agenda and for every majority-consistent -profile we have .
Applied to the preference agenda with the transitivity constraint, majority-preserving coincides with the requirement that a social welfare function should return the pairwise majority ordering whenever it is transitive; applied to the constraint, it coincides with the requirement that a social welfare function should return the pairwise majority ordering whenever it has a dominating element, i.e., whenever there is a Condorcet winner (which is slightly stronger than Condorcet-consistency).
mc, mcc, med, ra, leximax, y and mpc are majority-preserving. and are not even weakly majority-preserving.
Proof. Obvious cases are mc, mcc, med, ra, leximax, y and mpc. For , which coincides with the Borda rule for the preference agenda and the transitivity constraint, the result follows from the well-known fact that the Borda rule is not Condorcet-consistent. For , consider the profile in Table 3.
There are eight rational judgment sets over , i.e., . We check that for every , if then . Therefore, . Now, for every and for every . Therefore, although is majority-consistent and .
Let us call a pseudo-distance non-degenerate when there exist such that . Note that is non-degenerate.
For every distance , the rule is not majority-preserving. If is non-degenerate then is not weakly majority-preserving.
Proof. Let be two distinct judgment sets such that for all . Let . is majority-consistent, with , and yet contains also , therefore is not majority-preserving. If moreover is non-degenerate, let be three judgment sets such that . Let . is majority-consistent, with , and yet , therefore is not weakly majority-preserving.
is not weakly majority-preserving.
5 Inclusion relationships between the rules
We now establish the following (non)inclusion relationships between most of the rules introduced so far. As the case-by-case proof is long and not very interesting, we chose to have it in the Appendix.
Given two judgment aggregation rules and , we denote:
when holds for every agenda and every -profile .
when and .
when neither nor .
Let and be majority-preserving. Note that is not weakly majority-preserving, and that the counterexamples given in Section 4 all have an odd . If is odd (recall that is then a complete judgment set) then there is a majority-consistent profile for which , and . This implies that . Therefore, we have an incomparability relationship between a rule in and a rule in .
The inclusion and incomparability relations among the majority-preserving rules, and among the non majority-preserving rules, are represented on Tables 4 and 5; a sign for row and column means that , and ansign, that .
6 Unanimity, monotonicity, homogeneity, reinforcement
In preference aggregation, there are three classes of properties : those that are satisfied by most common rules (such as neutrality or anonymity); those that are very hard to satisfy, and whose satisfaction, under mild additional condition, implies impossibility results; and finally, those that are satisfied by a significant number of rules and violated by another significant number of rules. Similarly, in judgment aggregation, weak properties such as anonymity are satisfied by all our rules, while strong properties such as independence are violated by all our rules. We have already studied an “intermediate” property: majority-preservation. Here we consider four more: unanimity, monotonicity, homogeneity and reinforcement.
Unanimity has been defined for resolute rules by Dietrich and List : is said to satisfy unanimity when for every -profile and every , if for all , then .555A weaker unanimity property has been defined by List and Puppe , for resolute rules as well: whenever all the voters in have the judgment set . We first generalise unanimity to irresolute rules, which gives us a weak and a strong version of unanimity.
Definition 10 (Weak and strong unanimity)
Given , the -profile is said to be -unanimous when for every .
satisfies weak unanimity when for every -unanimous profile , there is a such that .
satisfies strong unanimity when for every -unanimous profile , for all we have .
mcc, med, , mpc, and do not even satisfy weak unanimity.
Voters + + + - + + - + + - + + - + + - + + - + + - - - - - - - - - m(P) + + - - + - - + - - Table 6: A profile showing that and do not satisfy weak unanimity.
, and coincide: . is obtained by reversing two judgments in either two of the three judgment sets of the profile.
: Let , where , and where and . selects all for which