Minimum Power to Maintain a Nonequilibrium Distribution of a Markov Chain
share
Biological systems use energy to maintain non-equilibrium distributions for long times, e.g. of chemical concentrations or protein conformations. What are the fundamental limits of the power used to "hold" a stochastic system in a desired distribution over states? We study the setting of an uncontrolled Markov chain Q altered into a controlled chain P having a desired stationary distribution. Thermodynamics considerations lead to an appropriately defined Kullback-Leibler (KL) divergence rate D(P||Q) as the cost of control, a setting introduced by Todorov, corresponding to a Markov decision process with mean log loss action cost. The optimal controlled chain P^* minimizes the KL divergence rate D(·||Q) subject to a stationary distribution constraint, and the minimal KL divergence rate lower bounds the power used. While this optimization problem is familiar from the large deviations literature, we offer a novel interpretation as a minimum "holding cost" and compute the minimizer P^* more explicitly than previously available. We state a version of our results for both discrete- and continuous-time Markov chains, and find nice expressions for the important case of a reversible uncontrolled chain Q, for a two-state chain, and for birth-and-death processes.
READ FULL TEXT