An Axiomatic Study of Scoring Rule Markets

by   Rafael Frongillo, et al.

Prediction markets are well-studied in the case where predictions are probabilities or expectations of future random variables. In 2008, Lambert, et al. proposed a generalization, which we call "scoring rule markets" (SRMs), in which traders predict the value of arbitrary statistics of the random variables, provided these statistics can be elicited by a scoring rule. Surprisingly, despite active recent work on prediction markets, there has not yet been any investigation into the properties of more general SRMs. To initiate such a study, we ask the following question: in what sense are SRMs "markets"? We classify SRMs according to several axioms that capture potentially desirable qualities of a market, such as the ability to freely exchange goods (contracts) for money. Not all SRMs satisfy our axioms: once a contract is purchased in any market for prediction the median of some variable, there will not necessarily be any way to sell that contract back, even in a very weak sense. Our main result is a characterization showing that slight generalizations of cost-function-based markets are the only markets to satisfy all of our axioms for finite-outcome random variables. Nonetheless, we find that several SRMs satisfy weaker versions of our axioms, including a novel share-based market mechanism for ratios of expected values.



There are no comments yet.


page 1

page 2

page 3

page 4


Budget Constraints in Prediction Markets

We give a detailed characterization of optimal trades under budget const...

Crowdsourced Outcome Determination in Prediction Markets

A prediction market is a useful means of aggregating information about a...

A New Understanding of Prediction Markets Via No-Regret Learning

We explore the striking mathematical connections that exist between mark...

Price Updating in Combinatorial Prediction Markets with Bayesian Networks

To overcome the #P-hardness of computing/updating prices in logarithm ma...

Log-time Prediction Markets for Interval Securities

We design a prediction market to recover a complete and fully general pr...

An Introduction to Artificial Prediction Markets for Classification

Prediction markets are used in real life to predict outcomes of interest...

Strategy-Proof Incentives for Predictions

Our aim is to design mechanisms that motivate all agents to reveal their...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The goal of a prediction market is to collect and aggregate predictions about some future outcome taking values in ; common examples arise from sporting, political, meteorological, or financial events. Prediction markets work by offering financial contracts whose payoffs are contingent on the eventually-observed value of . The agent’s choices are interpreted, by revealed preference, as predictions about , and thus the final state of the market is interpreted as an aggregation of agent beliefs.

Hanson [13] observed that one can design a prediction market using a proper scoring rule, which is a contract that scores the accuracy of prediction

(a probability distribution over outcomes) upon outcome

. The relevant guarantee is that the expected score is maximized when the agent reports their true belief . In Hanson’s scoring rule market (SRM),111We will use this term in lieu of the standard market scoring rule (MSR), as the latter could refer to either the scoring rule or the market mechanism. traders arrive sequentially and report their belief , and are eventually paid when is revealed.

In many cases, a market designer may only be interested in specific (e.g. summary) statistics, called properties, of trader beliefs, rather than the entire distribution over the outcome . Lambert et al. [15] observed that SRMs can be generalized to arbitrary scoring rules , wherein traders are paid , and if the expected value of is maximized by reporting the value of a particular property, in which case we say elicits the property, then the market should intuitively aggregate trader beliefs about the property in question. For example, one could design a “median market” by leveraging the fact that elicits the median of the distribution of , and we might expect the corresponding SRM to aggregate predictions about the median of .

Surprisingly, apart from exploring which properties are elicitable, practically nothing is known about these general SRMs apart from one property: expected values [13, 8, 2]. Here SRMs can be rephrased into a dual “cost-function-based” formulation where traders buy and sell shares in some securities , and the market prices reveal the trader’s belief about the expected value of (§ 2.25). The literature on prediction markets focuses on this setting, in which one can more easily study traditional market quantities like liquidity and depth.

In this paper, we step beyond expected values and study SRMs as a whole. Our first contribution is identifying which questions to ask; in particular we ask the following.

  1. [label=(0),itemsep=0pt,topsep=4pt]

  2. In what sense can we think of SRMs as “markets” in the traditional sense?

  3. Specifically, which SRMs behave like share-based markets, in the sense that traders can always “sell” their shares for some nontrivial price?

  4. Which SRMs if any maintain the known desirable characteristics of the cost-function-based framework, such as bounded worst-case loss and no arbitrage?

To answer these questions, we introduce axioms, and then study SRMs based on the property they elicit—as we will see, for example, markets eliciting modes will satisfy different axioms than those for medians, regardless of the scoring rules chosen. Our primary focus is the trade neutralization (TN) axiom, which captures question (2) above: traders holding a contract from the market maker can “sell” it for a nontrivial price, receiving more from transaction than the worst-case payoff of the contract. (Otherwise, traders would just keep their holdings regardless of their belief.)

Summary and results.

After reviewing prediction markets and scoring rules (§ 2), we introduce our axioms in §3

. We show that under some reasonably mild and standard axioms, incentive compatibility (IC) and path independence (PI), any mechanism that elicits predictions by offering contracts must be an SRM, thus justifying our focus on them. We then classify modes, finite properties, medians, quantiles, and expectations, according to what axioms their markets can satisfy (§ 


In § 5, we give our main result: any SRM satisfying trade neutralization must be a certain form of generalized cost-function-based market (Theorem 5.1). We note that unlike in prior work on cost functions [2], this result applies to any SRM for any property; so in a sense, it says that only close relatives of expectations can be elicited by any market satisfying trade neutralization.

Our exploration into non-expected-value SRMs yields a new prediction market mechanism worthy of attention on its own, for eliciting the ratio of expectations of random variables. As we describe in § 6, this new market framework exchanges a security (random variable) not for cash but for units of another security, thereby revealing trader beliefs about the ratio of their expectations.222To see this, note that a trader willing to “buy” one unit of in exchange for units of is effectively expressing the belief , i.e., . Similarly, selling at this “price” reveals the belief . Finally, we conclude in § 7 with a discussion of markets for other properties, elicitation complexity, and alternative market formulations. The Appendix contains all omitted proofs.

Relation to prior work.

While we discuss related work in § 2, it is important to distinguish our main result from prior characterizations of cost-function-based markets in the literature. In particular, as mentioned above, cost-function-based markets are known to be equivalent to scoring rules that elicit expected values [13, 8, 2]. In contrast, we ask which among a very broad family of mechanisms satisfy certain basic “market” axioms, and find that such mechanisms must fall into (a minor generalization of) the cost-function-based framework. Indeed, based on the above known equivalence, we can further conclude that our axioms imply the elicitation of expected values, though weakening them slightly allows us to elicit ratios of expectations (§ 6) and expectiles (§ E) while still retaining some sense of a “market”.

2 Aggregating Information with Prediction Markets

The goal of a prediction market is to crowdsource and aggregate beliefs of participants about some future event. It does so by allowing participants to buy and sell contracts which have different payoffs depending on the outcome of the event, and inferring a prediction from the participants’ choices. In this section, we formally define the class of such markets that we study, scoring rule markets (SRMs), with references to previous work. For other related work, see above and § 7.

2.1 Outcomes and contracts

Let denote the outcome space of interest to the market designer and a future event or random variable taking values in . The designer will in general allow traders to select contracts from a list offered by the market. The interpretation of a contract is that, when is eventually revealed, the market maker will pay the agent a net payoff (which may be negative).

Note that under this formulation, any initial payment the agent might make is folded in to . Specifically, letting denote the “all-ones” contract , then a contract could be written , interpreted as paying a price of now for the contract , whose payoff will be revealed when is observed. We assume agents are indifferent to timing of payments and just wish to maximize total expected payoff.

2.2 Automated market makers

When designing prediction markets, rather than a typical continuous double auction (“stock exchange”) mechanism, it is common to employ a centralized automated market maker, which offers to buy or sell any available contracts, and through which all trades are executed. (See [2] for practical reasons behind this design choice.)

Formally, a sequence of participants (traders) arrive at times and each selects a contract from a list offered by the market at that time. It will be convenient to let the set of contracts available be indexed at each time by some report space . Following Abernethy et al. [5], we consider a generic market making algorithm, termed a mechanism, that specifies the set of contracts available at each time. In general, this may depend on the entire past history of the market, and is represented as a mapping that, given a report of the participant, returns the corresponding contract. More formally, the contract given to a participant who chooses report given the past history of reports is denoted . The net payoff to the trader upon outcome will be denoted .

A popular instantiation of such a mechanism is the cost-function-based market maker, in which and , where is convex and  [2]. This payoff function corresponds to a trader making a fixed payment in return for a bundle of shares in the securities , i.e.  units of security , for an up-front cost paid to the market maker in terms of . Among the several nice properties of this market maker, one can see that a trader who believes has an incentive to buy or sell securities until , thereby revealing their belief.

2.3 Scoring rule markets for properties

The goal of the market is to incentivize a good prediction for some property or statistic of , such as the median or mean. Thus, much work considers prediction markets relying on proper scoring rules, which are contracts designed to elicit a single agent’s belief (i.e. the case of a market) [7, 12]. While originally designed to elicit an entire distribution over the outcome , in many cases, for example when is very large or even infinite, one may be interested in obtaining only summary information about this distribution. It is therefore natural to consider scoring rules which elicit such statistics, or properties, of distributions.

Here and throughout the paper, is a set of distributions of interest on the domain , for example, the distributions with full support, with finite expectation, or so on.

Definition 1.

A property is a function , which associates a set of correct report values to each distribution. We require to be non-redundant, meaning for any reports (i.e. we cannot have for all ). A property is single-valued if each maps to exactly one report, in which case we write as a function .

A scoring rule simply provides a payoff based on a reported value of the property and the observed outcome . We say the scoring rule elicits the property if the correct report is incentivized.

Definition 2.

A scoring rule elicits a property if for all , where . When is single-valued, this condition becomes . A property is elicitable if some scoring rule elicits it.

Not all properties are elicitable, meaning there is no way to score reports of their value based on an observed outcome in a manner which incentivizes truthfulness. A classic example is the variance of a distribution, which does not have convex level sets (mixtures of distributions with the same variance have higher variance in general), a necessary condition for elicitability 

[15]. (For such non-elicitable statistics, one could still discuss their elicitation complexity, the number of reports needed to compute the desired property post-hoc; we discuss this concept in § 7.) However, several well-known statistics are elicitable properties, including expected values and means, medians, quantiles, expectiles, and ratios of expectations.

Combining the concept of scoring rules for properties with scoring rule markets yields the following natural prediction market mechanism for an arbitrary property elicited by  [13, 15, 4]. Initialize the market state at some . When trader arrives, they can choose to update the market state to any , and once the outcome is revealed, the market maker will pay the trader . We can again express this mechanism using our notation above (we will later relate to ).

Definition 3.

A scoring rule market for scoring rule and initial state is the mechanism .

For brevity, we will simply write , or using contract notation , where of course .

3 Axioms and Preliminaries

To motivate our choice of SRMs, we consider two preliminary axioms in Appendix A that turn out to characterize SRMs; here, we briefly summarize. Incentive-compatibility (IC) states that agents maximize expected payment by choosing contracts that reveal their true belief about , for the property chosen by the market maker. (Without IC, it is not clear in what sense a market reveals useful information.) Path-independence (PI), an axiom appearing in prior work on prediction markets [2], ensures that a participant cannot gain more by making multiple trades in a row than by simply making the single optimal trade immediately. In Appendix A, we show that any mechanism which satisfies both PI and IC is an SRM for a property . It is conceptually similar to results for markets eliciting the mean [2, 5], but more general as it holds for any property.

Theorem 3.1.

A mechanism satisfies PI and IC for property if and only if it is a scoring rule based market (SRM) with some scoring rule that elicits .

Henceforth, unless otherwise noted we will assume PI and IC, which is to say we focus on SRMs.

3.1 Arbitrage and Bounded Loss

We now express two time-honored axioms in our notation: no arbitrage, and bounded worst-case loss. Recall that we write to mean the contract paying out for each . We also will write and to denote the payout bounds of contract ; note that we use infima and suprema as may be infinite, for example when .

The following axiom has been a desiderata since the inception of prediction markets: the market maker should not risk losing an unbounded amount of money.

Axiom 1 (Worst-Case Loss (WCL)).

An SRM , initialized at , satisfies WCL with bound if for all , .

As of course , WCL simply says that the scoring rule is bounded relative to the initial score .

Another classic condition for a market mechanism is that a trader should never be able to profit regardless of the outcome . In our contract notation, traders should never be able to purchase a contract for less than its minimum payoff , or equivalently, the net contract provided to traders should not be unconditionally positive.

Axiom 2 (No Arbitrage (ARB)).

SRM satisfies ARB if , .

The no-arbitrage condition was crucial in deriving cost-function-based markets in terms of convex conjugate duality [2]. Surprisingly, it turns out that any SRM satisfies no-arbitrage automatically if it is incentive-compatible.

Proposition 3.1.

SRM satisfies ARB if it is IC for some property .


If , then for all , . Thus, for any , , so cannot be an optimal report for any , contradicting non-redundancy (as ). ∎

3.2 New Axioms

We now identify several new axioms that capture desirable characteristics of prediction markets. The first is motivated by traditional markets, wherein a trader can always “neutralize” or “liquidate” their holdings. For example, if a trader buys a bar of gold, apart from apocalyptic scenarios, she can always sell it at any time for a strictly positive price. As another example, a trader holding an Arrow-Debreu contract, paying $1 upon event and $0 otherwise, should be able to sell it for some nonzero (perhaps very low) price. More generally, if the contract purchased is and is non-constant, the trader should receive strictly more than in cash (in both examples above, ). This condition is the trade neutralization (TN) axiom, which we now give along with two variants, one stronger and one weaker.

Axiom 3.

An SRM satisfies Trade Neutralization (TN) if for all trades , and all reports , there is a trade such that for some scalar .

Axiom 4.

An SRM satisfies Portfolio Neutralization (PN) if for all sets of trades , , and all reports , there is a trade such that for some scalar .

Axiom 5.

An SRM satisfies Weak Neutralization (WN) if for all trades , and all reports , there is a trade such that .

Portfolio neutralization (PN) is stronger than TN: for any set of purchased contracts, a trader should be able to sell the entire portfolio for at least its minimum payoff, all in one go, at any time. Finally, weak neutralization (WN) asks just that she be able to “make progress” by making some trade that improves her worst-case payoff. More precisely, WN states that traders holding a non-constant contract should be able to purchase a contract such that the minimum payoff of their portfolio strictly increases, i.e., . Note that WN is not far off from TN in the sense that a market maker allowing traders to exchange a contract for , i.e., “cash in” for its worst-case payout, then the mechanism would effectively satisfy TN. Naturally, we have PN TN WN; surprisingly, we will see that in fact TN PN (Theorem 5.1).

Note that the inequalities above are all strict. If we replaced them by weak inequalities, then WN would be trivially satisfied by essentially every conceivable mechanism: by just keeping your current contract , you are guaranteed a payoff of at least , by definition. This makes intuitive sense, as unless the contract is constant, it strictly dominates , so no rational trader would ever consent to trading for . The strict inequality thus captures a reasonable possibility for traders to “sell back” their previously-purchased contracts, in the sense that there is some belief a trader could hold where such a trade would be beneficial.

Our final axiom says that traders with (arbitrarily) small budgets can still expect to make money by participating in the mechanism.

Axiom 6.

A market satisfies bounded trader budget (BTB) if, for all market states and beliefs with , and for all , there exists a contract with and .

One can equivalently interpret BTB as stating that, as the overall market scales larger relative to the budget (or maximum allowable loss) of individual traders, they still have incentives to participate.

4 Central Examples

With our axioms in hand, we now turn to specific properties: the mode, median, quantiles, and expectations. For each, we wish to understand which axioms the corresponding SRMs can satisfy. As we will see, each of these properties has a unique signature with respect to our axioms.

4.1 Mode and Finite Property Markets

Perhaps the simplest example of any SRM is the canonical mode market for a distribution on outcomes , with report space and (“$1 iff you guess correctly”). Here we find that even the weakest version of neutralization, WN, is violated: a trade moving from to yields payoff , meaning this contract will lose the owner if . But if the current market state is, say, , then no report the agent can make will avoid this loss of when . For any report , the final payoff is . See Figure 1.

Figure 1: Visualizing trader “position” in a mode market. Each curve gives the payoff of a given scoring rule or contract as a function of the outcome . Here . Left: A trader who has reported stands to gain if and gain otherwise. Center: Similarly for . Right: A trader who moves the market from to gets the function . She stands to gain if , lose if , and gain otherwise.

Intuitively, the lack of WN has negative implications for agents, who must take on risk that cannot be mitigated later. It also violates BTB: The potential agent loss from any trade is a constant , so an agent with a budget smaller than will not be able to participate. This causes significant problems in practice for market designers as well, because the only possible solution, scaling down the scoring rule, is also unpleasant: many agents will be able to participate without much risk, so the market prediction will flip from outcome to outcome without necessarily aggregating information.

We emphasize that these characteristics are inherent to the mode, in the sense that they hold for any other scoring rule eliciting it. In fact, the same conclusions hold for markets eliciting any finite property , i.e. a property with a finite set of possible values.333See Lambert and Shoham [14] for motivation and examples for the finite case.

Theorem 4.1.

Any market for a finite property satisfies WCL, but not TN or BTB.

4.2 Mean and expectation markets

Consider now the mean of a random variable over the reals, .444In this setting, we naturally take to be the set of beliefs with well-defined expectations. As an illustrative example, consider the scoring rule , which elicits the mean. The corresponding SRM takes the form . Perhaps the first observation one makes is that, because , any nontrivial trade leaves the trader exposed to unbounded potential loss, as well as unbounded potential gain. This implies that BTB and WCL cannot hold.

What about trade neutralization? Consider a trader holding the contract for constants and . She would like to neutralize this position, so she must purchase some other contract of the form , so that her net position will be the constant . If the current market state is , our hero can neutralize her previous trade by choosing , so that . Her worst-case liability decreases from to a constant, showing that both WN and TN are satisfied. And in fact, even if she holds a set of contracts, her net position is simply the sum and can still be written in the form and the same argument goes through; this shows that PN is also satisfied.

Figure 2: Trader position in a mean market. . Left: Moving the market from to gives a position that pays off linearly in . Center: At , the trader chooses from a set of possible contracts, depending on if she reports , , etc. Right: choosing the contract that neutralizes the previous trade .

This example raises the questions of whether other markets eliciting the mean (if any) would have similar properties, and how this might depend on the random variable in question.

Motivated by this example, we now consider a much more general setting. Let be a given “random variable” and let be its expected value. Such a is called a linear property. We call SRM an expectation market if is IC for such a . For ease of exposition (i.e. to avoid relative interiors) we will assume that the range is full-dimensional in .

Capitalizing on the nice characterization of scoring rules for expectations due to [1, 10] (Theorem C.1), we know that any scoring rule eliciting takes the form


for strictly convex with subgradients and an arbitrary -integrable function. Note however that the term will vanish in the definition of , so without loss of generality we may take for all . We thus refer to expectation markets as being defined by if they satisfy Equation 1 for that (for any ). Letting be the convex hull of the set of outcomes of the random variable , we have the following characterization of worst-case loss for expectation markets:

Proposition 4.1.

The linear property has an expectation market satisfying WCL if and only if its domain is bounded, in which case the market defined by any bounded satisfies WCL.

Proposition 4.2.

The linear property has an expectation market satisfying BTB if its domain is bounded, in which case the market defined by any differentiable satisfies BTB.

To illustrate, let us revisit the example at the beginning of this subsection, i.e. and . (This corresponds to the convex function , up to a shift.) We saw that this market satisfies neither BTB nor WCL for , the entire real line. However, for , it satisfies both BTB and WCL, as and is bounded and differentiable on this domain. Finally, recall that it also (intuitively) satisfied TN, and furthermore, PN. This property can be shown to generalize, in particular, prior work has shown that linear properties have nice cost function based markets. Such markets take the form , for some convex function and . It then follows relatively directly that such markets satisfy PN, as any position (set of contracts held by a trader) can be written in the form . This can be interpreted as a fixed payment of and “shares” in a “security” . To neutralize, when the current market state is , it turns out to suffice to select the satisfying .

Theorem 4.2.

On a finite outcome space, i.e. , for any linear property there exists an expectation market satisfying PN, WCL, and BTB.

4.3 Median and Quantile Markets

We previously gave an example of a “mean market” given by , the scoring rule analog of squared loss . Perhaps the most natural statistic to investigate next is the median, elicited by the analog of “absolute loss”, . What we find is surprising: unlike squared loss, the absolute loss market does not satisfy PN, and in fact does not even satisfy WN. We show a general version of this result: no market for the median, or indeed any quantile, can satisfy WN. On the other hand, while the mean market could not satisfy WCL except on bounded domains, there are median and quantile markets that can.

Our setting in this section is as follows. Letting and , the -quantile of probability distribution with continuous CDF555This assumption is dropped in Appendix D. is the satisfying . Of course, the median is simply for .

Figure 3: Trader position in a median market. Here . Top left: The payoffs for and , and the contract that moves the market , namely . Top right: The same example but with with

the sigmoid function. Bottom: Two examples where a trader with position “red”

considers a potential contract “green” . If purchased, the net position will be the blue curve. It sometimes falls below the original position, meaning the trader’s worst case has gotten worse.

We first show that quantile markets do not satisfy WN: there may not be trades which improve the trader liability at all. Figure 3 gives an example with absolute loss.

Theorem 4.3.

No SRM for any -quantile satisfies WN.

Despite this negative result, quantile markets satisfy a surprisingly strong positive property. Recall that the squared loss market with , which elicits the mean of , could not hope to satisfy bounded worst-case loss if . And indeed, the absolute loss market shares this issue. There is an elegant work-around, however: use the sigmoid function , or another strictly monotone transformation, to map reports continuously into the interval . Then is still proper, as strictly monotone functions commute with the median: for any , all are mapped below and all are mapped above , so the quantiles are simply mapped as well.

Theorem 4.4.

For all , there is an SRM for the -quantile satisfying WCL.

Finally, quantiles also behave nicely with respect to bounded-budget traders. To see this for , notice that a small trade carries a liability of only , in the case . (See Figure 3.) But any trade smaller than can carry positive expected payoff for a trader if it moves the market closer to her belief.

Theorem 4.5.

If distributions in do not contain point masses, then any -quantile market satisfies BTB.

5 Characterizing Trade Neutralization

In this section, we will characterize mechanisms that satisfy trade neutralization (TN) when the set of outcomes is finite, . Recall that TN captures the quality of traditional markets, particularly in the sense of a “market maker”: the mechanism is always willing to buy back a contract it has previously sold, for a reasonable price.

TN may seem intuitively easy to satisfy, or at least might not appear to impose much structure on the market maker. In fact, we will see that the opposite is the case. To gain intuition for why TN might impose structure, recall that trades made by a participant must be interpretable as beliefs about the property the market is predicting. This applies to both the purchase of a contract and its sale back to the mechanism. Furthermore, this “canceling” trade must be available regardless of the current state of the market. We will leverage this intuition to show that the mechanisms satisfying TN are quite special and closely related to expectation markets.

To characterize the SRMs that satisfy TN, it will prove useful to separate out “contracts” from “cash” , even though technically both reside in the same space. In particular, we would like a canonical way to take a contract and separate out its “cash” component as , where intuitively is always paid regardless of the outcome, but depends on outcome. To do this, we simply define as the projection of

onto the hyperplane normal to


In general, for a scoring rule666Recall that given , one can define . We will indicate throughout where we assume IC. , we define its range , and define the corresponding “cashless” contract space by . Again, is simply the projection of the range of onto the hyperplane normal to . Similarly, we define to be the set of possible score differences modulo . Equivalently, is the projection of the range of onto the same hyperplane.

We now show that if an SRM satisfies TN, its corresponding set must have considerable structure: it must form an additive subgroup of .

Lemma 5.1.

If SRM satisfies TN, then is an additive subgroup of in the sense that implies . Moreover, the entire set of contracts is available at all times: for any , we have .


From the definition of TN, we must have the following: with for some . In terms of , as we have taken everything modulo , this implies such that . In other words, .

To show that

is a group under vector addition, we must show closure under addition and the existence of additive inverses. The latter is simple, as given any

we can write , so of course . We also have . For closure, consider , and write and . By the above, , so .

Finally, we must show that for any , we have . Clearly ; for the converse, consider . By the above, , and thus . ∎

For example, a construction allowed by Lemma 5.1 is (the integer lattice), and this is indeed possible. We give a example in Figure 4.

We will need the following generalization of standard cost-function based markets:

Definition 4.

The generalized cost-function based market parameterized by a convex “cost function” , “securities” , and “share space” is the SRM of the form with report space at market state given by .

Recall that in standard cost-function markets, , so that any trade is allowable. Furthermore, here is interpreted as the number of shares purchased of security . (For example, in the construction mentioned above, , we also have , so traders may only purchase integer numbers of shares in this example.)

Definition 5.

A cost-function based market defined by and is open if is differentiable and777We write for the interior of the set and for its convex hull. . It is quasi-open if for every pair of “share vectors” and subgradient , we have .

To see that quasi-openness is a relaxation of openness, we comment that if we required the condition to hold for all , then it would be equivalent to , which is a clear relaxation of openness; and quasi-openness requires somewhat less, as it only must hold for all .

Theorem 5.1.

Let SRM which is IC for be given. Then satisfies TN if and only if it is a generalized cost-function-based market for securities , where:

  1. [leftmargin=0.8cm,label=()]

  2. The share space of possible purchases is an additive subgroup of , and

  3. The market is quasi-open.

Moreover, TN implies PN.


For the forward direction, that TN implies cost-function-based and PN, we proceed in five parts: (1) construct a basis for , which will be the securities ; (2) rewrite in terms of that basis; (3) show that the map from reports to “shares” is bijective; (4) extract the cost function from the share representation of ; (5) show that satisfies our relaxed version of openness. For the converse, we will appeal to Theorem C.2.

1. Basis of .  By Lemma 5.1, is an additive subgroup of . Let be a basis for the linear span of . We can now write our securities in terms of this basis: define by . It will be convenient to work with in matrix form , which naturally we define as . Thus, we now have where is an additive subgroup of .

2. Rewrite .  By the definition of as the range of modulo , and the decomposition above, we know that for every we can write for some and . Thus, letting be the initial state of the given SRM, we have functions and such that for all , . By definition and thus we must have , meaning we can write . Note that the expected payoff when can now be written . In particular, can only depend on through , so letting as before, we have some such that .

3. The map is a bijection.  Surjectivity of follows from the fact that transformation by the change of basis matrix is a bijection, as by definition we are simply rewriting elements of via elements of . For injectivity, we will use IC. Suppose . If , is not non-redundant as , so clearly . Thus without loss of generality , which implies for any , so cannot be in the range of at all (i.e. the report dominates ), a contradiction. Thus, is injective and hence a bijection.

4. Extracting the cost function.  Now that we have a bijection from reports to “shares”, intuitively, we should just be able to take . More care is needed, however, as must be convex with the correct subgradients. Let us define a convex function by , which is the optimal expected score when . By IC and the definition of above, we have if and only if . Now we can define as the conjugate of , namely . Thus, we need only show that , as alluded to above, as this would imply as desired.

To make headway, we will appeal to convex analysis. First, we will establish that , meaning if is the optimal report for expected value , then is a subgradient of . This follows directly from the subgradient inequality; if is optimal for , then for all , , so . Conversely, suppose but is optimal for some , so . The subgradient inequality gives , which implies , so must be optimal for as well: .

In summary, we have . By [19, Thm E.1.4.1], we immediately have . Putting these together, we have . Thus, as every is in the image of , we must have for all .

5. is (almost) open.  We have established that can be written as a cost-function-based market with securities and a potentially nondifferentiable . To conclude this direction of the proof, suppose that for some we have . Following the proof of Theorem C.2, we start the trader at selling (as ) and place the state back at . To neutralize, the trader must now purchase , but by the same argument as before (using the fact that by construction of ), , which contradicts TN.

Converse.  Finally, for the reverse direction, we can simply repeat the proof of Theorem C.2: if the trades are elements of the group for all , then so is . It only remains to be shown that the price is less than the worst-case payoff of , which follows from integrating the guarantee in (ii) as in Lemma C.3. ∎

Figure 4: The “discretized” LMSR. This market for allows only integer trades of a single security, using the LMSR cost function and its conjugate . Traders may move the market from any “dot” to any other. The possible market states are equally spaced in share space (horizontal axis, left figure), translating to non-equally spaced prices (horizontal axis, right figure). Theorem 5.1 implies that all mechanisms satisfying PI, IC, and TN have this general format, being discretizations of cost-function markets.

This result shows that, despite the seeming weakness of the TN condition, the only SRMs satisfying TN are cost-function-based, or variations thereof (see Figure 4). The result also raises the question of whether any market can satisfy WN, or if perhaps WN is equivalent to TN in a formal sense. We will conclude by mentioning some interesting additional results answering this question and discussing future directions.

6 Share-Like Market for Ratios of Expectations

From Theorem 5.1, we know that any SRM satisfying the TN axiom will effectively be a cost-function-based market dealing in shares of some set of securities, perhaps restricting trades to some discretization of the share space. As a corollary, all such TN markets must elicit an expected value either directly or indirectly, or perhaps a discretization thereof. Theorem 5.1 leaves open the possibility, however, of a “share-like” market for a property other than an expected value which satisfies WN but not TN. We now give one such example: a ratio of expectations.

Before diving into the formalism, let us expand on the intuition given in § 1. In a cost-function-based market, traders purchase bundles of securities , in exchange for paying some up-front cost to the market maker in the form of cash. In other words, traders are exchanging the bundle of securities for an amount of the security . The ratio of expectations market can be thought of similarly, but now traders exchange the bundle of securities for an amount of some new security , which by convention will be nonnegative (and bounded away from for infinite ). In other words, the new market is exactly the same as the old, but the cost function is in units of the security rather than cash (). Immediately we can glean that, just as in a cost-function-based market traders with belief had an incentive to trade until , i.e. the change in cost is equal to the expected security payoff, now traders will trade until . Thus, the prices of the new market will be the market’s consensus about the ratio of expectations of and , .

What axioms does this new market satisfy? It would seem that its share-like nature could somehow circumvent Theorem 5.1 and satisfy TN, but of course this is not the case. As we will show, however, the ratio of expectations market does satisfy WN. First, we must formally define our new market.

Let be a finite set. For and with , we define the ratio of expectations . As usual we assume that is affinely independent, so is full-dimensional, and that , which since is finite is equivalent to the payoffs being nonnegative for all . A characterization of scoring rules eliciting was shown by Frongillo and Kash [10], which extended the real-valued case given by Gneiting [11]: a scoring rule elicits if and only if


where as in Theorem C.1, is strictly convex with selection of subgradients , and is arbitrary (and, as usual, irrelevant for SRMs).

Our main result for this section is that essentially all “open” markets for satisfy WN. The proof first applies convex conjugate duality to arrive at the cost-function-like market described above, and then applies facts about the usual cost-function-based framework (e.g. Lemma C.3) to show that the prices for trading in a bundle must be strictly less than its worst-case payoff.

Theorem 6.1.

A ratio-of-expectations market for differentiable and satisfies WN.


We construct a “cost function” as the convex conjugate of . The corresponding scoring rule is . By assumption, is a bijection, and thus markets with and with are equivalent. It therefore suffices to show that satisfies WN.

Consider any trade and let . For any , we will show that the trade satisfies WN. The difference in net payoff between the first and second trade is , so it suffices to show that this is always positive.

By the proof of Lemma C.3, we then have for any


Taking and , we have for all , which is equivalent to for all , thus establishing WN. ∎

By establishing WN for SRMs eliciting a ratio of expectations, we are essentially saying that these markets are “reasonable” and could plausibly aggregate trader beliefs effectively. It would be interesting to see how such markets perform in practice.

7 Discussion and Directions

We have presented market axioms for scoring rule markets (SRMs) and studied a handful of well-known statistics/properties to see which of these axioms their corresponding markets can satisfy. Interestingly, we have seen wide variation among the satisfied axioms, most dramatic of which are the neutralization axioms: TN is satisfied essentially exclusively by cost-function-based markets, whereas median/quantile and mode markets do not even satisfy its weakest version WN.

Perhaps the first question that comes to mind in light of our results is, are there any non-cost-function-based markets satisfying WN? We now give two positive examples: expectiles and ratios of expectations. As discussed in § 3, WN is essentially equivalent to TN, and thus these markets could be expected to perform similarly to cost-function-based markets. We then conclude with remarks and future directions.

Other properties.

We have studied the mean (expected values), median, mode, and ratio of expectations in this paper, but clearly many other properties of interest remain. One interesting example is the -expectile, a type of generalized quantile introduced by Newey and Powell [16], is defined as a solution to the equation . Scoring rules for expectiles take a hybrid form of those for means and quantiles, e.g. the asymmetric squared error, . We show in Appendix E that any SRM for expectiles satisfies WN, as the score is eventually linear, and thus can be neutralized by simply matching the slopes (analogous to Figure 2).

Other market forms.

Wolfers and Zitzewitz [20] study prediction markets from a more empirical perspective, and suggest a number of possible market formulations. One which is not covered here is to offer contracts such that . (This is called an identification function.) They illustrate this idea with a market eliciting the median of , where pays if and otherwise. While the properties of such a contract space would be interesting to study in a continuous double-auction setting, one may ask how to translate it to an automated market maker setting. Here the market would maintain a centralized median and traders could either buy or sell , moving . Unfortunately, for any , such a market has unbounded worst-case loss even on a bounded domain (traders buy and sell between and and ). For infinitesimal however, the market becomes the absolute-loss SRM with , because .


Finally, we remark that the recent concept of elicitation complexity brings interesting implications for our study. Here one asks, given a property of interest, how many dimensions does one need for there to exist an elicitable from some “nice” class of properties, where one can compute from via a link function , i.e. . In our context, one may choose “nice” to mean a property having an SRM satisfying any one of the axioms we discuss. The most natural may be TN, in which case one is essentially asking to be an expected value, a case studied by Agarwal and Agarwal [6]. It would be interesting to characterize properties having markets satisfying WN, and identify properties with low complexity with respect to these WN properties.


Thanks to Yiling Chen for helpful comments and pointers.


  • Abernethy and Frongillo [2012] J. Abernethy and R. Frongillo. A characterization of scoring rules for linear properties. In Proceedings of the 25th Conference on Learning Theory, pages 1–27, 2012. URL
  • Abernethy et al. [2013] Jacob Abernethy, Yiling Chen, and Jennifer Wortman Vaughan. Efficient market making via convex optimization, and a connection to online learning. ACM Transactions on Economics and Computation, 1(2):12, 2013. URL
  • Abernethy et al. [2014a] Jacob Abernethy, Sindhu Kutty, Sébastien Lahaie, and Rahul Sami. Information aggregation in exponential family markets. In Proceedings of the fifteenth ACM conference on Economics and computation, pages 395–412. ACM, 2014a. URL
  • Abernethy and Frongillo [2011] Jacob D. Abernethy and Rafael M. Frongillo. A collaborative mechanism for crowdsourcing prediction problems. In Advances in Neural Information Processing Systems 24, pages 2600–2608, 2011.
  • Abernethy et al. [2014b] Jacob D. Abernethy, Rafael M. Frongillo, Xiaolong Li, and Jennifer Wortman Vaughan. A General Volume-parameterized Market Making Framework. In Proceedings of the Fifteenth ACM Conference on Economics and Computation, EC ’14, pages 413–430, New York, NY, USA, 2014b. ACM. ISBN 978-1-4503-2565-3. doi: 10.1145/2600057.2602900. URL
  • Agarwal and Agarwal [2015] Arpit Agarwal and Shivani Agarwal. On consistent surrogate risk minimization and property elicitation. In JMLR Workshop and Conference Proceedings, volume 40, pages 1–19, 2015. URL
  • Brier [1950] G.W. Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3, 1950. ISSN 1520-0493.
  • Chen and Pennock [2007] Y. Chen and D.M. Pennock. A utility framework for bounded-loss market makers. In

    Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence

    , pages 49–56, 2007.
  • Frongillo and Kash [2014] Rafael Frongillo and Ian Kash. General truthfulness characterizations via convex analysis. In Web and Internet Economics, pages 354–370. Springer, 2014.
  • Frongillo and Kash [2015] Rafael Frongillo and Ian Kash. Vector-Valued Property Elicitation. In Proceedings of the 28th Conference on Learning Theory, pages 1–18, 2015.
  • Gneiting [2011] T. Gneiting. Making and Evaluating Point Forecasts. Journal of the American Statistical Association, 106(494):746–762, 2011.
  • Gneiting and Raftery [2007] Tilmann Gneiting and Adrian E. Raftery.

    Strictly proper scoring rules, prediction, and estimation.

    Journal of the American Statistical Association, 102(477):359–378, 2007.
  • Hanson [2003] R. Hanson. Combinatorial Information Market Design. Information Systems Frontiers, 5(1):107–119, 2003.
  • Lambert and Shoham [2009] Nicolas S. Lambert and Yoav Shoham. Eliciting truthful answers to multiple-choice questions. In Proceedings of the 10th ACM conference on Electronic commerce, pages 109–118, 2009.
  • Lambert et al. [2008] Nicolas S. Lambert, David M. Pennock, and Yoav Shoham. Eliciting properties of probability distributions. In Proceedings of the 9th ACM Conference on Electronic Commerce, pages 129–138, 2008.
  • Newey and Powell [1987] Whitney K. Newey and James L. Powell. Asymmetric least squares estimation and testing. Econometrica: Journal of the Econometric Society, pages 819–847, 1987. URL
  • Rockafellar [1997] R.T. Rockafellar. Convex analysis, volume 28 of Princeton Mathematics Series. Princeton University Press, 1997.
  • Savage [1971] L.J. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, pages 783–801, 1971.
  • Urruty and Lemaréchal [2001] Jean-Baptiste Hiriart Urruty and Claude Lemaréchal. Fundamentals of Convex Analysis. Springer, 2001. ISBN 978-3-540-42205-1.
  • Wolfers and Zitzewitz [2004] J. Wolfers and E. Zitzewitz. Prediction Markets. Journal of Economic Perspective, 18(2):107–126, 2004.

Appendix A Axioms

Our first axiom, incentive compatibility, is our formalization of the purpose behind designing the mechanism in the first place: it ensures that we can reasonably interpret the reports of the participants as predictions about the unknown outcome .

Axiom 7 (Incentive Compatability (IC)).

Given a property , a market satisfies IC for if for all histories , and all beliefs , we have