    # Bargaining with entropy and energy

Statistical mechanics is based on interplay between energy minimization and entropy maximization. Here we formalize this interplay via axioms of cooperative game theory (Nash bargaining) and apply it out of equilibrium. These axioms capture basic notions related to joint maximization of entropy and minus energy, formally represented by utilities of two different players. We predict thermalization of a non-equilibrium statistical system employing the axiom of affine covariance|related to the freedom of changing initial points and dimensions for entropy and energy|together with the contraction invariance of the entropy-energy diagram. Whenever the initial non-equilibrium state is active, this mechanism allows thermalization to negative temperatures. Demanding a symmetry between players fixes the final state to a specific positive-temperature (equilibrium) state. The approach solves an important open problem in the maximum entropy inference principle, viz. generalizes it to the case when the constraint is not known precisely.

## Authors

08/18/2019

### The Path to Nash Equilibrium

It has been proved that every non-cooperative game has a Nash equilibriu...
08/25/2020

### Entropy production and thermodynamics of information under protocol constraints

We investigate bounds on the entropy production (EP) and extractable wor...
12/22/2020

### Counter-terrorism analysis using cooperative game theory

Game theory has been applied in many fields of study, especially economi...
05/16/2022

### Thermodynamics as Combinatorics: A Toy Theory

We discuss a simple toy model which allows, in a natural way, for derivi...
11/18/2019

### Inverse Cooperative and Non-Cooperative Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning

Dynamic game theory provides mathematical means for modeling the interac...
11/02/2017

### Monotone bargaining is Nash-solvable

Given two finite ordered sets A = {a_1, ..., a_m} and B = {b_1, ..., b_n...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## References

• (1) I. Muller and W. Weiss, Entropy and energy: a universal competition (Springer Science & Business Media, 2006).
• (2) H.B. Callen, Thermodynamics, (John Wiley, NY, 1985).
• (3) L.D. Landau and E.M. Lifshitz, Statistical Physics, Part 1, Pergamon Press, 1980.
• (4) G. Lindblad, Non-Equilibrium Entropy and Irreversibility, (D. Reidel, Dordrecht, 1983).
• (5) G. Mahler, Quantum Thermodynamic Processes (Pan Stanford, Singapore, 2015).
• (6) J.F. Nash, The bargaining problem, Econometrica, 155-162 (1950).
• (7) R.D. Luce and H. Raiffa, Games and decisions: Introduction and critical survey (Courier Corporation, 1989).
• (8) A.E. Roth, Axiomatic Models of Bargaining (Springer Verlag, Berlin, 1979).
• (9) A.E. Roth, Individual rationality and Nash’s solution to the bargaining problem, Mathematics of Operations Research, 2, 64-65 (1977).
• (10) Supplementary material.
• (11) While our work is among first applications of game theory to statistical physics, connections between mathematical economics, thermodynamics and econophysics are well-known saslow ; candeal ; tsirlin ; smith ; yakovenko ; bruce ; e.g. there are interesting analogies between the axiomatic features of entropy and utility candeal . For application of statistical physics idea in evolutionary game theory see chatelier .
• (12) E.T. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106, 620 (1957).
• (13) Maximum Entropy in Action, edited by B. Buck and V.A. Macaulay (Clarendon Press, Oxford, 1991).
• (14) E.T. Jaynes, Where do We Stand on Maximum Entropy, in The Maximum Entropy Formalism, edited by R. D. Levine and M. Tribus, pp. 15-118 (MIT Press, Cambridge, MA, 1978).
• (15) J. Uffink, The constraint rule of the maximum entropy principle, Stud. Hist. Phil. Science B, 27, 47-79 (1996).
• (16) W. M. Saslow, An economic analogy to thermodynamics, Am. J. Phys., 67, (1999).
• (17) J.C. Candeal et al, Utility and entropy, Economic Theory, 17, 233-238 (2001).
• (18) S.A. Amel’kin, K. Martinaas, and A.M. Tsirlin, Optimal control for irreversible processes in thermodynamics and microeconomics Automation and Remote Control, 63, 519-539 (2002).
• (19) E. Smith and D. K. Foley, Classical thermodynamics and economic general equilibrium theory, Journal of economic dynamics and control, 32, 7-65 (2008).
• (20) V. M. Yakovenko and J. B. Rosser, Colloquium: Statistical mechanics of money, wealth, and income, Rev. Mod. Phys. 81, 1703 (2009).
• (21) B.M. Boghosian, Kinetics of wealth and the Pareto law, Physical Review E, 89, 042804 (2014).
• (22) A. E. Allahverdyan and A. Galstyan, Le Chatelier principle in replicator dynamics, Phys. Rev. E 84, 041117 (2010).
• (23) Axiomatic approaches to equilibrium thermodynamics have a long history; see callen ; buchdahl ; cooper ; lieb . They revolve around axiomatic introduction of entropy, whereas in our situation the entropy is just assumed to hold its standard form.
• (24) H. A. Buchdahl, The Concepts of Classical Thermodynamics, Am. J. Phys. 28, 196 (1960).
• (25) J.L.B. Cooper, The foundations of thermodynamics, Journal of Mathematical Analysis and Applications 17, 172 193 (1967)
• (26) E.H. Lieb and J. Yngvason, A Fresh Look at Entropy and the Second Law of Thermodynamics, Physics Today, 53, 32 37 (2000).

## I Supplementary Material

### i.1 1. Calculation of the minimum entropy for a fixed average energy

Here we show how to mimimize entropy

 Smin(E)=minpS[p],S[p]=−kB∑ni=1pilnpi, (16)

over probabilities

 {pi≥0}ni=1,∑ni=1pi=1, (17)

for a fixed average energy

 E=∑ni=1εipi. (18)

Energy levels are given.

Since is concave, its minimum is reached for vertices of the allowed probability domain. This domain is defined by the intersection of (17) with probabilities that support constraint (18). Put differently, as many probabilities nullify for the minimum of , as allowed by (18). Hence at best only two probabilities are non-zero.

We now order different energies as

 ε1<ε2<ε3..., (19)

and define entropies , where only states and with have non-zero probabilities:

 sij(E)=−E−εiεj−εilnE−εiεj−εi−εj−Eεj−εilnεj−Eεj−εi, (20) εi≤E≤εj. (21)

The minimum entropy under (18) is found by looking—for a fixed —at the minimum over all whose argument supports that value of . E.g. reads from (20) for (3 different energies):

 Smin(E)=min[s13(E), θ(ε2−E)s12(E)+θ(E−ε2)s23(E)], (22)

where is the step-function ( and ), and where we assume . Likewise, for :

 Smin(E)=min[s14(E),θ(ε2−E)s12(E) +θ(E−ε2)θ(ε3−E)s23(E)+θ(E−ε3)s34(E), θ(ε2−E)s12(E)+θ(E−ε2)s24(E), θ(ε3−E)s13(E)+θ(E−ε3)s34(E)] (23)

where . Generalizations to are guessed from (22, 23).

### i.2 2. Features of the Nash solution (9)

#### i.2.1 2.1 Concavity

Let us write (9) in coordinates (8):

 uN=argmaxu[us(u)], (24)

where (for obvious reasons) the maximization was already restricted to the maximum entropy curve . Now recall that is a concave function. Local maxima of are found from []:

 [us(u)]′|u=uN=uNs′(uN)+s(uN)=0. (25)

Calculating

 [us(u)]′′|u=uN=−2s(uN)uN+us′′(uN)<0 (26)

we see that solutions of (25) are indeed local maxima due to (concavity) and , , as seen from (8).

We shall now show that this local maximum is unique and hence coincides with the global maximum. For any concave function we have for

 s(u1)−s(u2)

which produces after re-working and using (25) for :

 us(u)−uNs(uN)

Hence is the unique global maximum of .

#### i.2.2 2.2 Comments on the textbook derivation of (9). Figure 4: Entropy-energy diagram in coordinates (8). Blue curve: s(u). The affine freedom is chosen such that the Nash solution (9) coincides with the point (1,1). The original domain of allowed states is filled in yellow. This domain is not symmetric with respect to s=u. This is seen by looking at the inverse function s−1(u) [red line] of s(u). The domain below AA1 is symmetric with respect to s=u.

In the main text we emphasized that the derivation of the solution (9) for the axiomatic bargaining problem that was proposed by Nash nash-1 and is reproduced in textbooks luce-1 ; rubin ; roth_book-1 has a serious deficiency. Namely, (9) is derived under an additional assumption, viz. that one can enlarge the domain of allowed state on which the solution is searched for. This is a drawback already in the game-theoretic set-up, because it means that the payoffs of the original game are (arbitrarily) modified. In contrast, restricting the domain of available states can be motivated by forbiding certain probabilistic states (i.e. joint actions of the original game), which can and should be viewed as a possible part of negotiations into which the players engage. For physical applications this assumption is especially unwarranted, since it means that the original (physical) entropy-energy phase diagram is arbitrarily modified.

We now demonstrate on the example of Fig. 4 how specifically this assumption is implemented. Fig. 4 shows an entropy-energy diagram in relative coordinates (8) with being the maximum entropy curve. The affine transformation were chosen such that the Nash solution (9) coincides with the point . Now recall (25). Once , then , and since is a concave function, then all allowed states lay below the line ; see Fig. 4. If now one considers a domain of all states (excluding the initial point ) lying below the line (this is the problematic move!), then is the unique solution in that larger domain. Moving back to the original domain and applying the contraction invariance axiom, we get that is the solution of the original problem.

### i.3 3. Maximum entropy method and its open problem

#### i.3.1 3.1 Maximum entropy method: a reminder

The method originated in the cross-link between information theory and statistical mechanics jaynes-1 . It applies well to quasi-equilibrium statistical mechanics grandy-1 ; balian-1 , and developed to become an inference method (for recovering unknown probabilities) with a wide range of applications; see e.g. grandy-1 ; balian-1 ; maxent_action-1 ; sbornik1 .

As a brief reminder: let we do not know probabilities (17), but we happen to known that they hold a constraint:

 U=U[p]≡∑nk=1ukpk, (29)

with

being realizations of some random variable

. We refrain from calling energy (or minus energy), since applications of the method are general.

Now if we know precisely the average in (29), then unknown probabilities (17) can be recovered from maximizing the entropy (16) under constraint (29). In a well-defined sense this amounts to minimizing the number of assumption to be made additionally for recovering probabilities maxent_action-1 . The outcome of the maximization is well-known and was already given by us in the main text:

 πi=e−βui/Z,Z=∑ni=1e−βui, (30)

where the Lagrange multiplier is determined from (29).

The method has a number of desirable features maxent_action-1 . It also has several derivations reviewed in grandy-1 ; balian-1 ; maxent_action-1

. Importantly, the method is independent from other inference practices, though it does have relations with Bayesian statistics

skyrms-1 ; enk-1 ; cheeseman_1-1 and causal decision making skyrms-1 .

#### i.3.2 3.2 The open problem

But from where we could know in (29) precisely? This can happen in those (relatively rare) cases when our knowledge is based on some symmetry or a law of nature. Otherwise, we have to know from a finite number of experiments or—within subjective probability and management science buckley-1 —from an expert opinion. The former method will never provide us with a precise value of , simply because the number of experiments is finite. Opinions coming from experts do naturally have certain uncertainty, or there can be at least two slightly different expert opinions that are relevant for the decision maker. Thus in all those case we can stick to a weaker form of prior information, viz. that is known to belong to a certain interval

 U∈[U1,U2],U1

This problem was recognized by the founder of the method jaynes-2 , who did not offer any specific solution for it. Further studies attempted to solve the problem in several different ways:

– Following Ref. thomas-1 , which studies the entropy maximization under more general type of constraints (not just a fixed average), one can first fix by (29), calculate the maximum entropy , and then maximize over , which will mean maximizing entropy (16) under constraint (31). This produces:

 maxp,U∈[U1,U2]S[p] = S(U1)forU1≥Uav (32) = S(U2)forU2≤Uav (33) = lnnforU1

Such a solution is not acceptable; e.g. in the regime (32) it does not change when increasing . I.e. the solution does not feel the actual range of uncertainty implied in (31).

– What is wrong with the simplest possibility that will state —i.e. the maximum entropy at the center of the interval—as the solution to the problem? Taking the arithmetic average of the interval independently from the underlying problem seems arbitrary.

Another issue is that each value of from the interval is mapped to the (maximum) entropy value making up an interval of entropy values. For this interval is , where and . Denoting by the inverse function of , we can take instead of .

– Ref. cheeseman_2-1 assumes that (though the precise value of the average is not known) we have a probability density for . Following obvious rules, the knowledge of translates into the joint density for maximum-entropy probabilities (30). While this is technically well-defined, it is not completely clear what is the meaning of probability density over the average . One possibility is that the random variable is sampled independently times ( is necessarily finite), and the probability density of the empiric mean

 1MM∑α=1u[α], (36)

is identified with . This possibility is however problematic, since it directly relates probability of the empiric mean with the probability of the average. E.g. if we shall sample independently times from (30), then the average of the empiric mean equals , but the empiric mean itself is not distributed via .

– Given (30), one can regard

as an unknown parameter, and then apply standard statistical methods for estimating it

rau-1 . Thus within this solution the maximum entropy method is not generalized: its standard outcome serves as the initial point for applying standard tools of statistics. This is against the spirit of the maximum entropy method that is meant to be an independent inference principle jaynes-2 .

– Yet another route for solving the problem was discussed in Ref. jaynes-2 . (We mention this possibility, also because it came out as the first reaction when discussing the above open problem with practioners of the maximum entropy method roger .) It amounts to the situation, where in addition to the empiric mean (36

) one also fixes the second empiric moment

as the second constraint in maximizing (16). It is hoped that since identifying the sample mean (36) with the average is not sufficiently precise for a finite , then fixing the second moment will account for this lack of precision. This suggestion was not worked out in detail, but it is clear that it cannot be relevant to the question we are interested in. Indeed, its implementation will amount to fixing two different constraints, i.e. in addition to knowing precisely the average in (29), it will also fix the second moment thereby assuming more information than the precise knowledge of entails. Figure 5: Entropy-energy diagram; cf. Fig. 1. Entropy is S and the minus energy U=−E for 4-level system with energies [cf. the discussion around (1)]: ε1=0, ε2=1, ε3=2.5, and ε4=10. Maximal (minimal) entropy curves are denoted by blue (black). All physically acceptable values of entropy and energy are inside of the domain bounded by blue and black curves. Red dashed line denotes ln2. Green dashed curve shows Uav; see (4). This figure illustrates the generalized maximum entropy method discussed in §3 and §4. Brown dashed lines indicate on specific values for U1 and U2 (U2

#### i.3.3 3.3 Solving the problem via bargaining

Main premises of this solution is that we should simultaneously account for both the uncertainty in and in , and that we should account for the duality of optimization, i.e. that the maximum entropy result can be also obtained via the optimization of under a fixed .

 U∈[U1,U2],Uav≡1n∑nk=1uk

The case is treated similarly to (37) with obvious generalizations explained below. The general case (31) is more difficult and will be addressed at the end of the next chapter.

We now know that the maximum entropy solution (30) can be recovered also by maximizing over for a fixed entropy. Note that the uncertainty (37) translates into an uncertainty

 S∈[S2,S1],S1≡S(U1),S2≡S(U2), (39)

in the maximum entropy. We now take the joint uncertainty domain in the diagram. includes all points , where [see (37)], and where ; see (39). has the structure of the domain required by Axiom 1, with the (initial) point

 (Ui,Si)=(U1,S2) (40)

being the unique anti-Pareto point, i.e. the unique point, where and jointly minimize. Recall the definition of the anti-Pareto set given in the caption of Table I.

We shall take Axiom 1 in a slightly restricted form as compared with its original form (Axiom 1) given by (6) of the main text 111The reason of this restriction is that for being able to deduce thermalization using only the restricted affine-covariance (42), we anyhow need to require the strict inequality ; cf the discussion after (8).

 Axiom 1′:Ui

Axioms 2 and 4 go on as stated in the main text. In particular, Axiom 2 ensures that is a convex domain, i.e. that it does not hit the minimum entropy curve.

However, the application of axiom 3 on the affine covariance needs a restriction, because it is seen from (40) that the initial point does transform in the affine way upon affine transformations of ; cf (7). On the other hand, is obviously intact under affine transformations of that we take as the restricted form of Axiom 3 222 We emphasize that for the present problem—where we have an uncertainty interval for —the inapplicability of the affine transformation is expected for at least two reasons. Firstly, the uncertainty interval does generally change under this transformation, which indicates on altogether a different problem. Secondly, the very definition of the initial point (40) does already connect with the maximum entropy curve; hence we do not expect the full freedom with respect to affine transformations to retain. :

 Axiom 3′:S→b−1S+c,b>0. (42)

It should be clear from our discussion in the main text—cf. discussions after (8) and after (9)—that Axioms 1’, 2, 3’ and 4 suffice for deriving thermalization. Note that the discussion after (9) of the main text assumed only affine transformations of only, but we could consider affine transformations of only with the same success; see Fig. 3. Note as well that instead of affine transformation (42) we can apply [i.e. we can reformulate axiom (42)], if the initial state (40) is parametrized as via the inverse function of .

Axiom 5 will go as stated in the main text and demands symmetry between maximizing and maximizing . Altogether, Axioms 1’, 2, 3’, 4 and 5 suffice for deriving

 argmax(U,S)[(U−U1)(S−S2)], (43)

as the solution of the problem with uncertainty interval (37).

#### i.3.4 3.4 Generalizing the solution to other types of uncertainty intervals

Uncertainty intervals that instead of (37) hold

 U∈[U1,U2],U1

are straightforward to deal with. Now relevant points on the maximum entropy curve can be reached by entropy maximization for a fixed , or minimizing for a fixed entropy . Hence the initial point and Axiom 1’ now read [cf. (40, 41)]:

 (Ui,Si)=(U2,S1), (45) Axiom 1′′:Ui>Uf,Si

 argmax(U,S)[(U2−U)(S−S1)]. (47)

Let us now take the case, where holds neither (37), nor (44), i.e. it holds:

 U∈[U1,U2],U1

Now we do not know a priori whether we should maximize or minimize over .

We define as above by joining together uncertainties of and , i.e. by including all points , where [see (48)], but where . The latter interval, due to (48), is where the maximum entropy values are contained. It is clear that generically has a structure of a right “triangle” formed by by two legs and a convex curve instead of the hypotenuse. Depending on whether or , we apply to either (40) with Axiom 1’ (41), or (45) with Axiom 1” (46). Axioms 2, 3’, 4, 5 apply without changes. Hence the initial point and solution reads, respectively in two regimes:

 ForS2

There is only one case, where solutions (50, 52) become umbiguous:

 S2=S(U2>0)=S1=S(U1<0). (53)

Indeed, now interpolating from

will lead to (50), which is different as compared with interpolating (52) from . Thus we state that under (53) the prior information (48) does not suffice for drawing a unique conclusion with the bargaining method.

Fig. 5 illustrates all the above solution on an entropy-energy diagram. Two examples of the set are shown by blue, green (dashed) and red (dashed) lines. Brown dashed lines show an example of (), where the domain of states allowed according to Axiom 1 is not convex: it will cross the minimum entropy curve denoted by black lines on Fig. 5. Hence such values of are not allowed. Fig. 5 shows two examples of allowed intervals : and . The corresponding values of and are and . Blue arrows join the initial states with corresponding Nash solutions (43, 47).

It is seen that for , where , there are two Nash solutions denoted by dashed magenta arrows on Fig. 5. For this case (53) the existing prior information does not allow to single out a unique solution.

## References

• (1) J.F. Nash, The bargaining problem, Econometrica, 155-162 (1950).
• (2) R.D. Luce and H. Raiffa, Games and decisions: Introduction and critical survey (Courier Corporation, 1989).
• (3) M.J. Osborne and A. Rubinstein, Bargaining and markets (Academic Press, 1990).
• (4) A.E. Roth, Axiomatic Models of Bargaining (Springer Verlag, Berlin, 1979).
• (5) S. Karlin, Mathematical Methods and Theory in Games, Programming and Economics (Pergamon Press, London, 1959).
• (6) E.T. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106, 620 (1957).
• (7) W.T. Grandy, Jr. Foundations of Statistical Mechanics. Vol I: Equilibrium Theory (eidel, Dordrecht, 1987). Foundations of Statistical Mechanics. Vol II: Non-equlibrium Phenomena (eidel, Dordrecht, 1988).
• (8) R. Balian, From Microphysics to Macrophysics, Vol. I, (Springer Science & Business Media, 2007).
• (9) Maximum Entropy in Action, edited by B. Buck and V.A. Macaulay (Clarendon Press, Oxford, 1991).
• (10) Maximum-Entropy and Bayesian Methods in Science and Engineering. Volume 1: Foundations; volume 2: Applications, edited by G. J. Erickson and C. R. Smith (Kluwer, Dordrecht, 1988).
• (11) J. J. Buckley, Entropy Principles in Decision Making Under Risk, Risk Analysis, 1, 303 (1985).
• (12) B. Skyrms, Updating, supposing, and MaxEnt, Theory and Decision, 22, 225-246 (1987).
• (13) S.J. van Enk, The Brandeis Dice Problem and Statistical Mechanics, Stud. Hist. Phil. Sci. B 48, 1-6 (2014).
• (14) P. Cheeseman and J. Stutz, On the Relationship between Bayesian and Maximum Entropy Inference, in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, edited by R. Fischer, R. Preuss, and U. von Toussaint, American Insititute of Physics, Melville, NY, USA, 2004, pp. 445 461.
• (15) E.T. Jaynes, Where do We Stand on Maximum Entropy, in The Maximum Entropy Formalism, edited by R. D. Levine and M. Tribus, pp. 15 118 (MIT Press, Cambridge, MA, 1978).
• (16) M. U. Thomas, A Generalized Maximum Entropy Principle, Operation Research, 27, 1188-1196 (1979).
• (17) P. Cheeseman and J. Stutz, Generalized Maximum Entropy, AIP Conf. Proc. 803, 374 (2005).
• (18) J. Rau, Inferring the Gibbs state of a small quantum system, Phys. Rev. A 84, 012101 (2011).
• (19) R. Balian, private communication.